'It should be laid to rest. We have nothing to do with any of that, and it's just sad that because we're neighbors, you know, you get thrown into stuff,' Gloria explained. 'The truth will rise to the top'