Jeremiah Brown - who was one of the most loved stars of the reality show before he was voted off by his fellow castmembers - has addressed the offscreen racism controversy