Elle kept a low profile as she opted for dark sunglasses and walked alongside Doyle, who dressed casually in brown pants and a khaki jacket