Despite being known for her edgy outfits during her pop heyday, Kesha switched up her outfit by slipping into a revealing blue dress