Although Joseph's siblings reportedly don't have any desire to get to know him, he has continued to make overtures