At the time, the conservative - now the center's chairman - said the move was designed to put an end to events featuring performers in drag