'In the beginning, she refused to join him because she didnt want to be seen as one of those girls that is in and out of a A-list celebritys life,' dished the insider