He has hit out at critics Brett has called antisemitic in interviews and social media posts over the years