Michael Fortier, with his daughter Émilie, stands beside Christian Paradis and James Moore during the cabinet swearing-in ceremony at Rideau Hall in Ottawa. (Fred Chartrand/Canadian Press)

Quebec played a key role in Wednesday's mini-federal cabinet shuffle, with two Conservatives from the province taking on larger portfolios.

Senator Michael Fortier, a Montrealer who doesn't have a seat in the House of Commons, was appointed international trade minister.

Christian Paradis, an MP for the southeastern riding of Mégantic-L'Érable, was picked to take over Fortier's former job as public works minister. Paradis will also continue in his role of secretary of state for agriculture.

"We're on track. We're going stay on track," Prime Minister Stephen Harper said after the new ministers were sworn in at Rideau Hall in Ottawa.

He offered no further comments, and his newly appointed ministers were also tight-lipped.

B.C.'s Moore takes on Olympics, languages duties

The shuffle was triggered by Maxime Bernier's sudden resignation as foreign affairs minister on May 26.

David Emerson, considered an experienced and reliable minister, was asked to fill in at foreign affairs temporarily, and on Wednesday he was appointed to the role. He had been minister of trade but will vacate the position now that Fortier is stepping in.

Gov. Gen. Michaëlle Jean and Prime Minister Stephen Harper sing the national anthem at Wednesday's Rideau Hall ceremony. (CBC)

Emerson was flying to Japan on Wednesday for a meeting of foreign ministers and was not present at the swearing-in ceremony.

In one final move, James Moore, a bilingual parliamentary secretary from the British Columbia riding of Port Moody-Westwood-Port Coquitlam, was named secretary of state for official languages, the Asia-Pacific gateway and the Vancouver 2010 Olympics.

Moore rose to prominence when the so-called Chuck Cadman affair came to light earlier this year. Harper asked Moore to answer almost all opposition questions in the House of Commons about allegations that the Conservatives had once bribed Cadman, an Independent MP who died three years ago of cancer.

Conservatives want to increase seats in Quebec

Leading up to Wednesday, speculation was swirling that Quebecers would play a central role in the shuffle. The Conservatives, who are hoping to win more seats in Quebec in the next election, were expected to promote Quebecers into cabinet to make up for the loss of Bernier.

Quebec's business community welcomed the role the province will play in the federal cabinet. Françoise Bertrand, president of Quebec's Federation of Chambers of Commerce, said she is glad to see Paradis taking on more responsibilities.

"He certainly is not as experienced as Mr. Fortier or Mr. Emerson, but certainly he has proven that he can play an active role which he had been doing as the secretary of state during the first two years."

Bertrand said the move shows the prime minister's confidence in Fortier, and the importance of Montreal within cabinet.

Harper last shuffled his cabinet in August to move Bernier to foreign affairs from industry.

Bernier, who remains an MP for the riding of Beauce, south of Quebec City, resigned after admitting he had left classified documents at the home of his former girlfriend, Julie Couillard, who has had ties to criminal biker gangs.

Bernier is scheduled to speak publicly for the first time about the events that led to his resignation on Wednesday at 6:30 p.m. ET. He will give a speech to his constituents in Beauce but will not be taking questions from the media.