The inseparable couple's street sighting came the day after Taylor's most recent ex-boyfriend - 1975 frontman Matty Healy - took a bitter dig at her album The Tortured Poets Department while onstage performing at the Glastonbury Festival in England