A source has now revealed exclusively to the Daily Mail that de Ramon didn't want to join Pitt at star-studded events at first because she feared it would discredit her jewelry company