Sam looked ready to hit the greens wearing a pair of tan khakis with a navy blue golf shirt and white sneakers