Plum Point was known as Old Ferolle, but the name of the community was changed to Plum Point in 1931 to distinguish it from New Ferolle.
The community was used by the Basque as a base for preparations when retuning home from the summer fishery. It was later used by the French as a fishing room. In 1884, there were 30 settlers but the area was later abandoned and the first recorded settler was Henry Coombs in 1897. The residents relied on the fishery, mainly cod, lobster and herring.
In 1928, the International Pulp and Paper Company began a pulpwood cutting operation in the area. The population of the town remained small until the 1960's when some families from Current Island and St. Genevieve Bay resettled here.
Along with the fishery, Plum Point has become a small service centre for the area.