Spark

Technology and unintended consequences

We're not very good at predicting the potential side effects of our tech

Sometimes the digital cure is worse than what it's trying to fix

Sometimes the result is very different from the goal. (Adam Killick)
Listen to the full episode53:59

We humans are great at creating new technologies designed to make our lives easier. 

Unfortunately, these creations are often fueled by hubris and a remarkable inability to foresee unintended consequences. 

From bagging your own groceries at the self-checkout, to the destructive power of just one Yelp reviewer, to all the "shadow work" we don't realize we're taking on,  the effects of our digital activities are often impossible to predict.

"Technochauvinism"

Meredith Broussard is quick to point out that she's not a technophobe.

Her book is called Artificial Unintelligence: How Computers Misunderstand the World. The book is described as a guide to understanding the inner workings and outer limits of technology.

From the first time she built a toy robot when she was a kid, she realized that using tools and building technology could be a lot of fun. She still loves technology because its amazing possibilities.  

Meredith Broussard (Lucy Baber)
 

But Broussard's book examines a flawed assumption that she calls "technochauvinism." Technochauvinism is simply the belief that technology is always the solution to every problem. 

Broussard started her career as a computer scientist and is now a data journalist. She's also a data journalism professor at the Arthur L. Carter Journalism Institute at New York University. Throughout her career she's viewed technochauvinism from many different perspectives. 

"It comes from a particular kind of bias that says that mathematical and engineering problems are superior to human problems," Broussard said. 

Technochauvinism creates a disconnect between what technology is capable of and what we think it can do. 

"That disconnect is very, very, real," added Broussard, but one of the ways we can address the problem is by disassembling that technology.

"By taking things apart and putting them back together, you can learn how they work, and you can learn that it's not magic."  

However, Broussard is concerned that because computer science and the tech sector are descendants of mathematics, they have inherited the same biases. She believes the concept of technochauvinism comes from the idea that the mathematics is the most crucial thing, and nothing else matters.


Related: How shadow work chews up our time


"That's why we see things in Silicon Valley like the code is the thing that matters, let's move fast and break things, let's disrupt.

"But culture matters, social issues matter, and they matter just as much as solving mathematical and engineering problems." 

But it's not just mathematics and an overly technical approach that can lead to unintended consequences. There's also the role of emotion.

Yelp reviewers as managers

Take restaurant reviews, for example.

Joshua Sperber believes that amateur restaurant reviewers don't always have enough insight into why there are issues with the service. 

Sperber is an Assistant Professor of Political Science & History at Averett University and the author of a the book, Consumer Management in the Internet Age: How Customers Became Managers in the Modern Workplace.

Sperber researched the impact of online review sites on restaurants. For his book, he conducted interviews with wait staff, restaurant managers, and Yelp reviewers. 

"We know that servers have been fired for reviews. We know that servers have been suspended for bad reviews. It adds stress, this knowledge that any customer is a potential Yelp reviewer," said Sperber.


Related: I am not your Uber driver


He argues that while online reviews may seem on the surface to be a great way to empower patrons, those reviewers may unwittingly be putting restaurant workers under a kind of surveillance.
Joshua Sperber

Sperber said that Yelp gives restaurant employees "a reason to assume they're likely being watched by someone who is not just going to grumble and complain about a supposed bad experience but is actually going to write it up on a website read by millions of people including their manager."

While he found Yelp reviewers to have a strong "consumerist mentality," he said many were unaware of the direct impact negative reviews can have on restaurant employees. 

"It gives [owners] a justification to increase worker productivity and to punish one worker, but more importantly sort of scares the rest of the workers into working harder because they know they could be punished based on a negative Yelp review," said Sperber.