There was plenty of baking, swims in the pool and an adorable group photo, curated like a Vogue spread, which Swift described as a 'family portrait' on Twitter