His ex-girlfriend, actress Evan Rachel Wood (pictured together in 2006), also accused him of sex crimes but never filed a lawsuit, with Manson vehemently denying all accusations