Harrison was 60 and Calista was 38 when they first started dating, and many were quick to slam the unlikely pairing (seen in 2003)