The superstar performer dressed casually for her visit, wearing black tights and a white knit dress