Search and rescue officials in Nova Scotia have called off the formal search for five missing fishermen, saying rough seas and cold water have diminished any chance of survival for the men.

The search for the crew of the Miss Ally, a 13.5-metre boat that overturned off the southern coast of Nova Scotia, has now been turned over to the RCMP as a missing persons case.

George Hopkins, the father of one of the missing fishermen, says he knew all too well the chances his son, Joel, would be found alive diminished with every passing hour.

As the search wound down "everybody got quiet here [in Woods Harbour]."

Crew members haven't been seen since the boat ran into trouble in rough weather about 120 kilometres southeast of Liverpool late Sunday night.

Two Canadian Coast Guard vessels, a Hercules aircraft, a Cormorant helicopter and a private plane contracted by the Department of Fisheries and Oceans spent Tuesday scanning an area of 11,265 kilometres off Nova Scotia's shore.

A U.S. Coast Guard plane also joined the effort before the search was scaled back.

George Hopkins sits at the head of the table waiting for news about his son Joel. (CBC)

Most of the fishermen, whose names have not been released, are in their 20s.

"All them boys, we love them," said Hopkins breaking down in tears. "They're the heart of the community.

"We live on the water. We don't leave the wharf with a fear in our heart. There's no fear there. It's like a miner going down to the mine," Hopkins said. "It's our lifestyle."

Hope fades

Friends and family of the missing men held an anxious vigil while waiting for news on Tuesday.

Many people in Woods Harbour expressed anger at the economic conditions that led to the men winter fishing in the first place.

"It's the most dangerous time of the year for the fishery because of the weather," Hopkins said Tuesday. "The seas get so high and the water is so cold."

The Miss Ally went missing about 120 kilometres southeast of Liverpool. (Canadian Press)

On Tuesday, he sat with friends looking at a map of the area and figuring out where a life-raft might have drifted to. His second son, Jesse, was also fishing on the weekend, but his captain brought their boat into harbour during the storm. He was back home in Woods Harbour on Tuesday.

The CBC's Phonse Jessome reported that all of the men are well-respected in the community and several have children.

Hopkins said Joel got sick the first three times he went fishing with his dad, but he was undaunted.

"Joel loved fishing," his father said. "He started when he was eight or nine years old. Even if he got seasick, it didn't faze him at all."

He described Joel, who had recently become a father himself, as a man with a love for life.

"Joel was always a kid that lived on the edge, no matter what he did. If he was riding his four-wheeler or dirt bike, he was always on the edge. He wanted to do everything to the limit, and fishing was no different."

George Hopkins is no stranger to sea accidents — he was on two fishing boats that sank before he turned 26.

George Hopkins described his son Joel, who had recently become a father himself, as a man with a love for life. (Facebook)

Unsure if people on life-raft

The U.S. Coast Guard first spotted what it believed to be the life-raft of the boat Sunday night and dropped a buoy to mark the site.

"They dropped a data-marker buoy, which provides a surface temperature and a sea state," said Petty Officer Clinton Dotson of the U.S. Coast Guard. "It's a very large area it's been drifting over."

Waves were reported at seven to eight metres on Monday and six metres on Tuesday.

The U.S. searched with an HC-144A Ocean Sentry, a twin-engine turbo prop with an internet connection to transmit pictures directly to the Joint Rescue Co-ordination Centre in Halifax.

Dotson said the first U.S. assistance came from the crew of the HU25 Falcon on Sunday. Poor visibility meant they could not see the boat, but the infrared camera picked out what they thought was the life-raft.

"If there were actually people on board the life-raft," he added, "but that we could not tell. We were unable to determine whether the people had gotten into the life-raft."

Although the formal search is over, ships travelling off Nova Scotia's coast have been asked to be on the lookout.