The former One Direction star, 31, partied into the early hours with pals at Worthy Farm on Saturday, and was seen getting very close to a woman in the VIP area