The actor, 61, said he was approached to be on the show at first to play a love interest of Kim Cattrall's character Samantha who was curious about black men's 'anatomy'