The royal wore a short-sleeved burgundy button down shirt for the outing, paired with black leather riding gloves and his typical white-striped riding helmet