The couple started as they meant to go on with the hand holding and even at the reception of Queen Elizabeth's coffin at Westminster Hall they were spotted hand in hand