The two actors previously sparked romance rumours following their respective splits, however a source poured cold water on the claims when speaking to TMZ