Matt has faced backlash of his own after he recently starred in an ad for the beauty brand E.l.f. Cosmetics