Caspar split from Ellie, 38, (pictured in June 2023) after nearly five years of marriage last year, is an investor in the restaurant allowing him to date Olivia in relative privacy