The Emmy-nominated actor, 66, explained on Wednesday to Us Weekly his rationale in not reprising the role of Trey MacDougal, the ex-husband of Kristin Davis' character Charlotte York on the HBO series