'I gotta figure out like ways to make sure that I do have things that are just between us,' she said; The duo seen in 2024