Ne-Yo has previously defended his lifestyle, saying 'can't see how it's hurting anybody', while also hitting out at the fact polygamy is illegal