Canada's prime minister has replaced embattled Veterans Affairs Minister Julian Fantino with retired air force officer and first-time MP Erin O'Toole.

In a release Monday, the government said Fantino will remain in cabinet as associate minister for defence — the same post he held before being named international co-operation minister in 2012.

This time, Fantino will focus on Arctic sovereignty, information technology security and foreign intelligence, the release said.

The change was made during a quiet ceremony at Rideau Hall around midday and addresses for now what has been a nagging controversy for Prime Minister Stephen Harper.

O'Toole is a former member of the Canadian Forces and one of the founders of the True Patriot Love Foundation, which raises money to support military families.

MP Erin O'Toole

Erin O'Toole leaves Rideau Hall in Ottawa on Monday after being sworn in as minister of veterans affairs. (Sean Kilpatrick/Canadian Press)

As he left Rideau Hall, O'Toole told reporters, "It's a pleasure to serve our veterans."

Fantino frequently criticized

Fantino faced repeated opposition calls for his resignation or firing in the fall over his handling of the Veterans Affairs portfolio. The department has faced much criticism from some veterans because of the decision to close regional offices and for a lack of support for veterans with mental illness.

In November, the auditor general found the department was not doing enough to provide mental-health services to veterans, just days after it was revealed the government had returned nearly $1 billion in lapsed funding to the treasury in recent years.

Fantino was out of the country attending commemorative Second World War events as the opposition called for a response to the auditor general's report.

Fantino was roundly criticized for a testy meeting with veterans early last year and for refusing to speak with the wife of a veteran who pursued him down a hallway in Parliament.

In an earlier attempt to address the problems at Veterans Affairs, Harper named Walt Natynczyck, a popular former chief of defence staff, as deputy minister in October.

The Royal Canadian Legion, which represents veterans and boasts 300,000 members, welcomed the appointment of O'Toole, saying in a release it hoped the change would bring "better action" to the issues facing veterans.

"This is a political move and we will not concern ourselves with the reasons behind it," Tom Eagles, Dominion president of the legion, said in a release.

Harper had defended Veterans Affairs handling

In an interview with CBC News chief correspondent Peter Mansbridge last month, Harper was asked whether he remained confident in Fantino as minister.

"By definition the prime minister has confidence in all of his ministers," the prime minister told Mansbridge.

He defended the government's handling of Veterans Affairs, but conceded that "as time has gone on, it’s become apparent that there are some gaps" in programs.

"We will respond and make the changes we need to make where we see real gaps in the services," he said.

NDP Leader Tom Mulcair challenged O'Toole to reopen nine Veterans Affairs offices the government closed a year ago and said previous ministers have simply carried out the prime minister's policies.

"That's the tragedy of the Conservatives. They talk a good game, they love having their picture taken on Nov. 11 [Remembrance Day], but the rest of the year they're not there. You've got to be there all day long."

Fantino 'pleased' by progress in 2014

Mulcair said it has been a terrible year for Fantino and that it's a mystery to him how he remains in cabinet.

"It's a half-hearted firing of an incompetent minister," Mulcair said.

Liberal Leader Justin Trudeau issued a written statement on Monday putting the blame for the problems at Veterans Affairs squarely on the prime minister.

“It is clear that Mr. Harper is wholly responsible for his government’s failing of our veterans.

"Shuffling Mr. Fantino into another cabinet role will, regrettably, not fix this shameful situation," Trudeau said in a written statement on Monday.

In an op-ed column that ran in Sun Media papers on the weekend, Fantino wrote he was "pleased at the progress we have made over the past year [in Veterans Affairs] and am fully committed to seeing further progress and improvement as we move into 2015."

O'Toole was elected to the House of Commons in Durham, east of Toronto, in a 2012 byelection to replace Bev Oda, another former minister who left cabinet amid controversy. Since 2013, he had served as parliamentary secretary to International Trade Minister Ed Fast.

O'Toole is a graduate of Royal Military College in Kingston, Ont., and served in the Royal Canadian Air Force until 2000. He also has a law degree from Dalhousie University in Halifax.

His father, John O'Toole, was the Progressive Conservative member of the Ontario Legislature for Durham from 1995 until last May.