Serena Williams (pictured while appearing in Netflix's docuseriesHarry & Meghan) was also reportedly absent from the Duchess' 44th birthday celebrations