'She's neither a vegan or an influencer,' said the actor, who separated from his wife, Lulu Williams, in 2023 (pictured together in 2019)