Joe tied the knot with Sofia in 2015, when she was at the height of her Modern Family superstardom and he had become a sex symbol via True Blood and Magic Mike