Celebs have also entered the scene including fashion designer Tommy Hilfiger, whose sprawling mansion (pictured) sits right outside the beach