At one point, the performer talked about the track Only Love from his album Music For People Who Believe In Love, which dropped on May 23; Joe seen in 2024 in NYC