Islamic tradition states that the Zamzam Well was revealed to Hagar (Hājar), the second wife of Abraham and mother of Ishmael. By the instruction of God, Abraham left his wife and son at a spot in the desert and walked away. She was desperately seeking water for her infant son, but she could not find any, as Mecca is located in a hot dry valley with few sources of water. Hagar ran seven times back and forth in the scorching heat between the two hills of Safa and Marwah, looking for water. Getting thirstier by the second, the infant Ishmael scraped the land with his feet, where suddenly water sprang out. There are other versions of the story involving God sending his angel, Gabriel (Jibra'il), who kicked the ground with his heel (or wing), and the water rose. A similar story about a well is also mentioned in the Bible.

The name of the well comes from the phrase Zomë Zomë, meaning "stop flowing", a command repeated by Hagar during her attempt to contain the spring water.

According to Islamic tradition, Abraham rebuilt the Bayt Allah ("House of God", cognate of the Hebrew-derived place name Bethel) near the site of the well, a building which had been originally constructed by Adam (Adem), and today is called the Kaaba, a building toward which Muslims around the world face in prayer, five times each day. The Zamzam Well is located approximately 20 m (66 ft) east of the Kaaba.