The singer was spotted strolling through the bustling city and at one point, she also stood closely next to her eldest son