Fledgling couple Harry and Taylor dated from November 2012, with a source telling The Sun at the time he was 'head over heels' for her