The name of the community comes from its resemblance to the harbour of Le Kerpont near St. Malo, France. The spelling of the name of the community has varied: Carpon, Carpunt, and Karpoon and it is pronounced to rhyme with harpoon.
The community has the most northerly sheltered harbour on the island portion of the province. Jacques Cartier may have visited the harbour and it was one of the centres for the French fishery on the French Shore. Captain James Cook charted the area.
The community was one of the earliest sites for a meeting between the Inuit and the Moravian missionary Jens Haven, in 1764. In the 1800s English fishermen began to come to the area to fish.
One of the first permanent English settler was Fred Pynn, who was a "gardien" of the French premises there in 1872. The first census was taken in 1857 and showed a population of 69. Until the 1880s French fishermen outnumbered the Newfoundland fishermen in the area. During this time the community became the last stop for vessels on their way to the summer fishery on the Labrador coast.
By the 1920s this fishery had come to a halt and the community became a base for the resident inshore fishery. When the road was completed to the community in the 1960s, 47 residents relocated to nearby larger areas and by 1970, the residents of the settlements of Little Quirpon, Fortune, and Jacques Cartier Island had resettled there