He also explained why they split up for 'three or four years' back then before they found their 'way back' to each other in the home they raised their daughter Ella Olivia Stiller, 23, and son Quinlin Dempsey Stiller, 20; pictured in December 2013