The famous Kalahari Bushmen of southern Africa have long been in decline. For more than a century, the people, who speak Khoisan languages, have been pushed off their land by farmers and brutalized by colonialists.

Yet for tens of thousands of years, the Khoisan’s ancestors were members of “the largest population” on the planet, according to a new study.

Published online today in Nature Communications, biochemist Stephan Schuster of Nanyang Technological University in Singapore and colleagues sequenced the complete genomes of five Khoisan hunter-gatherers from Namibia and compared them with the DNA from 1462 genomes of people from around the world.

Schuster’s team found that two of the Khoisan, members of the Ju/’hoansi population in Namibia, inherited their DNA only from Khoisan ancestors in the northern Kalahari region and showed no sign of interbreeding with non-Khoisan speakers. These two Ju/’hoansi genomes preserve ancient diversity inherited entirely from their direct ancestors.

Using several different methods of analysis, the team reconstructed population sizes for the ancestors of the Khoisan, as well as for Europeans, Asians, and another African group, the Yoruba. They found that all four groups declined in effective population size (the number of breeding adults) between 120,000 and 30,000 years ago. The non-Khoisan groups’ numbers plunged precipitously—by 30,000 years ago, European and Asian populations had plummeted by 90% from their peak, thanks to population bottlenecks caused by the migration of small groups out of Africa. But the Khoisan population declined by only 26%.

Until 20,000 years ago, when Bantu farmers spread south, the Khoisan may have been the most populous group on the planet.