'If you would've asked me then, will you celebrate your 50th wedding anniversary? I would've said, Yeah, if we live that long, absolutely. I was just so sure,' he told People; Pictured in Beverly Hills in March 2018