The Town takes its name from its early origins of Seldom Come By. This name was given to the harbour because vessels travelling the East Coast of Newfoundland and Labrador would seldom come by without stopping, especially in stormy seas when they would weather the storm in the deep, sheltered harbour.
Seldom is well known for the Burnt Point Lighthouse which was built in 1905 and operated manually until 1992 when it was replaced by a automated system.
Seldom was established in 1700 and Little Seldom in 1857. Settlers have strong religious ties with Anglican and Pentecostal churches still active in the community. St. Augustine's Anglican church was built in 1890 and is still being used for worship today. Both churches are open to the public during the day in summer months.