Some fans have also speculated that Swift may be on the outs with Margaret Qualley, who is the wife of Swift's frequent producer and collaborator Jack Antonoff