Politics

Canada election 2015: Online polling defended, while some still question reliability

A polling expert says the industry is going through difficult times because fewer people are willing to participate in surveys, and adds citizens reading about polls should remember they are only snapshots of a particular moment in time.

Response rates can be low and results misleading, especially during an election campaign

Craig Worden of polling firm Pollara Strategic Insights says surveys focusing on the election horse race between the Liberals, NDP and Conservatives this early in the campaign are 'pretty ludicrous.' (CBC)

Election campaigns are notoriously unpredictable but one thing is certain: Canadians will be bombarded with public opinion polls until the federal vote on Oct. 19.

But how accurate and representative are the data?

Statisticians and survey experts say the industry is going through upheaval as an ever-diminishing number of people are willing to participate in polls, and there is no longer a reliable and complete registry of citizens that can be used to randomly select people for research.

Moreover, the number of polling firms has increased significantly over the past several years, creating a "Wild, Wild West" environment for public opinion research due to the various kinds — and quality — of methodologies used to collect data, according to Andrea Perrella, director of the Laurier Institute for the Study of Public Opinion and Policy.

The polling industry is going through "dark days," he said.

No single methodology

"Fewer people are answering surveys," Perrella explained. "There is no longer a single methodology and even in the most carefully administered telephone polls, response rates are so low we are not too sure what those non-respondents are thinking."

Gone are the days when the vast majority of citizens had landlines and firms had reliable lists of people they could randomly call.

Today, firms need to spend more money to include cellphone users in phone surveys, while other companies build online panels of people who volunteer to be solicited to answer questions online.

Pollsters aware of 'near crisis'

Cliff Zukin, professor of public policy and political science at Rutgers University, penned an op-ed in the New York Times in June where he said "election polling is in near crisis, and we pollsters know."

In an interview, Zukin said online, panel-based polls are problematic because most are not probability-based, meaning people aren't selected randomly because there is no complete list of voters' IP Internet addresses.

"[Online panels] are starting with a badly selected sample," he said. "It's not a sample of anybody, it's people who have volunteered to take polls — that's not a good sample of the population."

He said the only way to get a truly representative sample for an online poll is to randomly select voters and give laptops to people who don't have Internet access in order for them to complete the survey — which a handful of U.S. firms are doing — at a high cost.

Fewer landlines

Moreover, as a result of low response rates and the decrease in the number of landlines, Zukin considers a telephone poll reliable only if 50 per cent of the sample includes people reached on their cellphone.

Sebastien Dallaire of the Leger polling firm disagrees with Zukin on the reliability of online panels.

Leger boasts a panel of 400,000 households from which it draws to conduct its online surveys.

But online databases growing

While Dallaire says "not all panels are created equal," he maintains Leger has created a strong database that reflects the Canadian population.

Poll results so far in the election campaign show a kind of horse race between (from left to right) the Conservative Leader Stephen Harper, NDP Leader Tom Mulcair and Liberal Leader Justin Trudeau. At least one pollster says those polls are only useful as a general snapshot of where people stand at a point in time when they aren't engaged in the campaign and won't be for several weeks. (CBC)

"Our entire reputation rests on the fact we can produce quality results that are a [near] match with actual voting on voting day," he said.

"If we couldn't do this it would be terrible for us. If we didn't believe in it we wouldn't be using (panels) because our success rests on the trust people have of what we do."

Dallaire wouldn't use the words "dark days" or "crisis" to describe the industry but said it was "going through a transition."

"Response rates have diminished and it poses a problem," he said. "From that point of view it's something that everyone is thinking about."

Too early to get solid results

But aside from the tumult rocking the industry, experts question the usefulness of the high number of polls that have already been released in the early stages of an 11-week election campaign.

Craig Worden of polling firm Pollara Strategic Insights, said surveys focusing on the election horse race between the Liberals, the NDP and the Conservatives this early are "pretty ludicrous."

"At this point the polls [that have come out] are useful as a general snapshot of where people stand at a point in time when they aren't engaged in the campaign and won't be for another month and a half," he said.

Perrella said horse-race polling is problematic for another, more serious reason.

Pick issues over horse-race approach

"The horse-race approach makes voters want to pick a horse and that's not really the intent of our democratic discourse," he said.

He said polls should ask many questions, focus on underlying trends and try to offer voters a detailed and in-depth understanding of how Canadians think about important issues.

Perrella added that horse-race polling turns voters off the idea of coalition governments or a more collaborative approach to politics.

"Coalitions are the backbone of democratic consensus-building but a horse race is not about a consensus," he said.

"It's about a single winner and all the others are losers. That kind of frame limits the ability of people to look at another party as little more than an adversary."

Comments

To encourage thoughtful and respectful conversations, first and last names will appear with each submission to CBC/Radio-Canada's online communities (except in children and youth-oriented communities). Pseudonyms will no longer be permitted.

By submitting a comment, you accept that CBC has the right to reproduce and publish that comment in whole or in part, in any manner CBC chooses. Please note that CBC does not endorse the opinions expressed in comments. Comments on this story are moderated according to our Submission Guidelines. Comments are welcome while open. We reserve the right to close comments at any time.