Hailey looked downcast as she stepped out that night to Sushi Park, an exclusive West Hollywood restaurant she frequents with Justin and her friends