Calls are growing for Prince Andrew to be kicked out of the Royal Lodge (pictured), the lavish Windsor mansion he shares with his ex-wife