As they hopped in a taxi to head home he sweetly put his arm around her back following the busy night