After fans spotted that the duo have removed photos of each other from social media, they quickly started speculating that their brief romance had already ended