Byrne was chic wearing a tan-colored vest that was buttoned at the front as well as a pair of matching trousers