Spark

Weapons of Math Destruction

'What big data does is, it segments our society into winners and losers'
(Michelle Parise)
Listen18:45

Cathy O'Neil has memories of being a little kid clutching her Rubik's Cube on her way home from math camp and gazing out the car window to study the numbers on license plates. For her, math was a refuge from the messiness of the real world.

Mathematician Cathy O'Neil

Today she's a mathematician, she writes about quantitative issues on her blog mathbabe.org, and is the author of a new book, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy.

Cathy's on a bit of a crusade against algorithmic math models that she believes create injustice and reinforce discrimination.

"Overall, what big data does, and which it does very well is, it segments our society into winners and losers," she says "and the losers are the usual suspects: the people who are poor, people who are badly educated, people of colour, people with mental health problems."

Cathy first realized how the misuse of mathematics could hurt people when she spent some time working in finance. What she witnessed in the global financial crisis in 2008 convinced her we've given way too much power to algorithms. Cathy has a theory about why the lessons of 2008 didn't make us rethink the power we give algorithms in our society.

"I think people just didn't get the message. They hated the bankers, they looked for the villains in the banking industry, and I don't really blame them for that," Cathy says "but one of the things I kept wondering, is why do we invest so much faith and trust now in the algorithms in the world of big data, when we already know that mathematical algorithms could be basically shielding corruption? We should have learned that lesson, but didn't."

So how do we change this trajectory we seem to be on? For starters, Cathy believes we need to start thinking about, and teaching, ethics to those in the field of data science.


We've covered the power of - and problems with - algorithms quite a bit on Spark. One of the reasons is because of its relationship to the privacy of our data. You may not be doing anything "wrong", but data about your behaviour can still be used against you.

If you want to learn more about algorithms, privacy, and bias, you can check out these previous Spark stories: Frank Pasquale on The Black Box Society, Suresh Venkatasubramanian on algorithmic bias, Christian Sandvig on algorithm awareness, and Brian Christian on his book Algorithms to Live By (check out the video below!)

Brian Christian, co-author of Algorithms to Live By, explains how algorithms can help us make major life decisions. 1:01

Comments

To encourage thoughtful and respectful conversations, first and last names will appear with each submission to CBC/Radio-Canada's online communities (except in children and youth-oriented communities). Pseudonyms will no longer be permitted.

By submitting a comment, you accept that CBC has the right to reproduce and publish that comment in whole or in part, in any manner CBC chooses. Please note that CBC does not endorse the opinions expressed in comments. Comments on this story are moderated according to our Submission Guidelines. Comments are welcome while open. We reserve the right to close comments at any time.