Gosselin's father Jon was granted full custody of him in 2018 and his 21-year-old sister Hannah in 2016, whom he still talks to on occasion (pictured in 2023)