Are polls bad for democracy?
A poll conducted for the British paper The Sun made international headlines recently, but probably not for the reasons its authors expected.
The poll was reported with the startling headline that "1 in 5 British Muslims have sympathy for Jihadis." But it turns out that the people polled were only asked if they had sympathy for young British Muslims joining fighters in Syria; it did not specify which side the fighters were on.
Polling companies have also been quick to speak up and question the survey's methodology but the incident has triggered bigger questions about polls regarding where they fit, if at all, in democracy and in the public conversation.
Mario Canseco, Vice President of Public Affairs at Insights West — a market research firm operating in BC and Alberta — says it's better for pollsters to pursue and promote public insight than to let someone with a vested interest fill that gap.
The full interview is available in the audio player above. The following portions have been edited for clarity and length.
Q: What does this Sun story say about polling?
A: I think there are several complications with that particular survey. One of them, obviously, is the fact they chose the word 'sympathy' to describe something; it's not a word I would use on a specific scale, if I was doing a survey of this magnitude. And the other one is that it has been misinterpreted and misquoted consistently, not only by the media that commissioned it, but also by others. There are a lot of stories out there that mention this is sympathy towards ISIS, which is not what they asked. In this industry, you have be not only careful about the questions you are asking but also the kinds of answers that you allow people to have. If you force somebody to say something that is not going to be representative of the entire population — I think this is what happened, for this particular survey.
Q: I've also heard it suggested that respondents are often being asked to comment on something that they may have never given any thought to at all. Should you be screening respondents to test their knowledge on the subject, before they're asked to weigh in?
A: That has always been one of the great questions of the industry. If you're looking at a survey, let's say of Canadians, you need to ensure that you're meeting the census targets: that you're speaking with this many men, this many women, this many young adults, middle-aged adults, people over 55, this many people in Ontario and Quebec and Alberta — and you need to make sure they have a chance to say, 'You know, I haven't really read a lot about this'... When you force people to say 'yes' or 'no' to something, that's when you get bad data.
Q: Your firm does research on a wide range of topics, from government policy to consumer trends. This week, for example, you sent out results from a survey on how British Columbians and Albertans feel about Black Friday. What's the point of a poll like that?
A: We like to have a balance of policy issues... but also have something that is fun, interesting and engaging... We want to make sure that we have the right balance and figure out which groups are more likely to be partaking in shopping on Black Friday, who don't enjoy it, who might be willing to do it online, just to have that read because you're surrounded by so much information and it's difficult to make sense of it, unless you have something that is properly balanced.
Q: And I'm sure it's a little bit of marketing for Insights West, as well?
A: Oh, absolutely. One of the best things we can do is measure public opinion, to talk about these things. The only way to know whether you're getting things right is to call elections correctly. Unfortunately our industry went through a really bad situation: the election of 2012 in Alberta, the election in BC in 2013, the election in Ontario in 2013, where some of the forecasts didn't really work very well. And the only way to get back is to continue doing this. If we decide not to do polling and not to ask these questions, then somebody else who has a vested interest in the outcome is going to be the one publishing data and I don't think that's the Canada where we want to live.
Click the blue button above to listen to the full interview.