Prime Minister Stephen Harper's director of communications, Dimitri Soudas, announced Wednesday he will be leaving his job this fall.

Soudas is a long and loyal member of Harper's staff, and first started working for the prime minister nine years ago.

He started as an aide to Harper when he was Opposition leader, and after Harper became prime minister in 2006, Soudas worked as a press secretary. Soudas was promoted from associate director of communications to chief spokesman in the spring of 2010. The Montreal native also served as an adviser to Harper on Quebec.

Soudas, 31, publicly announced his departure on Twitter and said his first priority when he finishes the job on Sept. 5 will be his wife and three young children.

Soudas made the announcement shortly after the Conservative caucus met  on Parliament Hill. It was the party's first meeting since winning a majority mandate on May 2.

Following the meeting, Environment Minister Peter Kent said the job of communications director is "brutal" and is one that Soudas performed well. "It's very demanding, day and night without exception," he said.

"It will be a tough search," Kent said about finding a replacement for Soudas. "They are very large shoes to fill, Dimitri did that job very well."

His colleagues in the Prime Minister's Office said they are sad to see him go but wish him well in the next phase of his life.

"He has served this prime minister with unmistakable loyalty, passion and dedication," Sara MacIntyre, Harper's press secretary, said in an email. "Those that have worked closely with Dimitri know of his great personal sacrifice, tenacity and perseverance."

Soudas is known as a very hands-on and influential member of Harper's staff. A fierce defender of the prime minister, Soudas has at times had a rocky relationship with the national media.

He also found himself the subject of controversy on certain occasions, most recently during the spring election campaign, when he fended off allegations that he tried to influence the appointment of a new head for the Montreal Port Authority.

Just days ago, Soudas accompanied Harper on a trip to Greece where they visited the village of Soudas's grandparents. Part of the tour included a visit to the scene of a Second World War massacre by the Nazis. Soudas's grandfather was among those killed.

Soudas was Harper's fifth director of communications since 2006. He took over from John Williamson who only held the job for a few months. Williamson quit so he could focus on running as a Conservative candidate in New Brunswick. He won and is now among the new members of Parliament.

Before Williamson, Kory Teneycke was Harper's chief spokesman and before him, Sandra Buckler had the job.