The former One Direction star was accompanied by a group of friends while making his way across the Worthy Farm site during a weekend of partying