Prime Minister Justin Trudeau says his government's proposed tax changes aren't worrying most Canadians — except for those affected, who he said are making "a lot of noise."

Trudeau sat down with Here & Now's Debbie Cooper on Tuesday and said he's had a lot of conversations with people in Newfoundland and Labrador since arriving Monday, and added few people have brought up the proposals.

"A lot of those wealthy folks are really fighting to keep those benefits that they have, and they're making a lot of noise," he said.

The government has been consistently saying it needs to make sure the middle class doesn't pay more in taxes than it needs to, he said.

People who make less than $150,000 won't be affected

"That also applies to middle-class small businesses — fishers, farmers, people who work extraordinarily hard and contribute to growing the economy. We just want to make sure that people using private corporations don't have benefits that aren't available to average Canadians, and that's where we're making a little tweak."

'The system as it's set up gives too many benefits to wealthy people and not enough to the people who really need the support.'  - Justin Trudeau

In response to criticism from small-business owners that the changes appear punitive, Trudeau said the government is listening to concerns, and added people making less than $150,000 won't be affected negatively by the changes.

"At the same time, we're trying to ensure that folks who are successful are paying their fair share, and it's not fair that people who are wealthy have tax rules that they can get out of paying as many taxes and as much tax as middle-class families, and that's something that goes to the heart of fairness."

Trudeau said the government isn't suggesting that people who have benefited from the current system are doing anything wrong.

N.L. will benefit from Canada-Europe free trade, says PM

"We're just reflecting on the fact that the tax system as it's set up gives too many benefits to wealthy people, and not enough to the people who really need the support," he said.

Asked what the federal government is doing to help Newfoundland and Labrador experience the same economic growth being experienced in the rest of Canada, Trudeau was light on specifics but mentioned $100 million for the Atlantic Fisheries Fund and working closely with Premier Dwight Ball to improve innovation and immigration in the province.

"When we talk about the benefits of the Canada-Europe free trade agreement, the proximity of Newfoundland to Dublin, to continental Europe as compared to the rest of the country, is going to be a tremendous advantage here," he said.

You can watch the full interview in the video player above.