Fin previously joined Garner and dad Ben Affleck, 52, as well as brother Samuel, 13, at the Boston Red Sox game in July