According to Scorpia, some think it is just a myth. Many consider being sent there a fate worse than death.
The stories Adora remembers from when she was a Horde conscript say the place has blood beetles and trees with knives in place of leaves. She shivers as she remembers stories of “chippits with razor-sharp teeth, scruffers with razor sharp horns, and razor-fins with razor-sharp teeth… really not sure how they got that name…” However, long after she defects to the Rebellion, Shadow Weaver admits that those were only “children’s stories” about the place, which while true, masks the true dangers of the place, which are worse. The island's greatest danger is a signal that the island emits constantly. Whoever hears it is caught by their own fears and insecurities until they become part of the island.
In Season 4, it is revealed that the First Ones dumped all their secrets on the island inducing their plans about the Heart of Etheria Project. It was a dumping ground for old First Ones tech. Some of it was useless but some of it still worked like the signal.
It might a been a whole ship, but was used for dumpage.