About our analysis of intolerant language

About our analysis of intolerant language

We commissioned media marketing company Cision to analyze the use of intolerant language by Canadians online between the months of October 2015 until November 2016. This search included a 1% representative sample of all Twitter mentions, thousands of web forums, blogs, and news media websites. This search contained 40 derogatory terms that are either racist or religiously or ethnically offensive. Cision segmented these mentions further into two groups, one focused on white supremacist language, the second on Islamophobia, to provide a better picture of the types of intolerant language being used by Canadians, how it has changed over time and the types of discussions that include this language.

There were three ways Cision filtered this data to provide a realistic and accurate picture of racist language use:

1. Contact filtering
To the best of its ability, Cision removed social media comment sources who were clearly providing commentary on the issue of race. As an example, Shaun King, a writer with the New York Times, will occasionally use terms which land his comments in Cision's search, and Canadian commenters sharing these comments without adding additional text would not be counted in this analysis. This ended up being a list of more than 100 journalists, comedians and musicians who were clearly not being racist, and often have content shared broadly by Canadian social media users.

2. Hand-scored sampling
The second way Cision filtered the data was through a hand-scored review of 500 comments in both the month of November 2015 and November 2016. In November 2015, 8 per cent of the comments tracked were those of commentary or when comments were used ironically. In November 2016 commentary accounted for 10 per cent of the mentions. With that sample completed, Cision applied a 10 per cent control to the entire analysis.

3. Bot removal
Not all social media commentary is created by real social users. In many cases some of the most prolific users in a discussion are automated bots, retweeting or sharing specific types of comments written by specific accounts. Cision removed four commenters who created more than 3,000 comments during the analysis period that were bots. More filtering could be done; Cision could, for instance, increase the sample rate or further widen the list of contacts to separate out. However, the company is confident in the breath and quality of the mentions collected as an indicator of increased intolerance online.

Cision did a hand-scored deletion but the volume of mentions would require months, or hundreds of hours of analysis so the company did a sample and applied a control.

Here's what the report found: