Affleck, who recently celebrated his 53rd birthday with the teens, appeared relaxed and smiled while looking dapper in a white T-shirt with tan pants and a black and gray jacket