The original settlement, Kaole, was founded c. 800 CE, and grew into an important trading town by the 13th century. The Kaole Ruins contain the remnants of two mosques and 30 tombs, dated back to the 13th century. Until the 18th century, Bagamoyo, the settlement 5 kilometres (3 mi) north of Kaole, was a small trading center where most of the population were fishermen and farmers. Their main trading goods were fish, salt and gum, among others.[2] Around the 17th century this area began growing in prosperity and by the 18th century it was an important stop in the caravan trade, acquiring the name Bagamoyo. It became the most important trading entrepot of the east central coast of Africa in the late 19th century.

In the late 18th century, Muslim families settled in Bagamoyo, all of which were relatives of Shamvi la Magimba in Oman. They made their living by enforcing taxes on the native population and by trading in salt, gathered from the Nunge coast north of Bagamoyo. In the first half of the 19th century, Bagamoyo became a trading port for ivory and slave, with traders coming from the African interior - places as far as Morogoro, Lake Tanganyika and Usambara- on their way to Zanzibar. This explains the meaning of the word Bagamoyo ("Bwaga-Moyo") which means "Lay down your Heart" in Swahili. It is disputed whether this refers to the slave trade which passed through the town (i.e. "give up all hope") or to the porters who rested in Bagamoyo after carrying 16-kilogram (35 lb) cargoes on their shoulders from the Great Lakes region (i.e. "take the load off and rest"). There is considerable debate regarding the extent of the slave trade as a major export in Bagamoyo, with archival analysis suggesting that ivory was the primary export over slaves, and that many of the caravan porters on the ivory route were free wage laborers as opposed to slaves.[3][4] However, the history of the slave trade features prominently in the shared culture of its residents and organizations such as UNESCO emphasize its importance as a cultural heritage site memorializing the slave trade in East Africa.[5][6]