Zayn - who was accused of infidelity several times during his relationship with Perrie - did not address the rumours at the time (pictured with ex-fiance Perrie)