Bantry Bay is part of the traditional lands of the Garigal people, who made extensive use of the area for tens of thousands of years before the arrival of European settlers. Numerous rock engraving sites and shell middens, some at least 4,600 years old, can still be seen in the area as tangible evidence of this Aboriginal past. It is likely that Aboriginal people continued to use the area for fishing and shellfish collecting until they were gradually dispossessed by European settlement and succumbed to disease.