The actor, 58, known for his oversized outfits, wore a navy blue polo shirt tucked into a grey pair of suit trousers as he stepped out