Title: Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy
Author: Cathy O’Neil
My rating: 4.5/5

Many of my classmates from grad school have found jobs as data scientists, others have become Wall Street quants. PhD physicists are often hired for these data science/big data jobs because we have the statistical and computer programming skills for the job. As more and more quantitative (or otherwise computable) data becomes available, algorithms and data science are becoming an ever more important part of our lives. Rather than ask, “is this a good thing?”, perhaps the better question is “what are the downsides?”

This is exactly the question that O’Neil addresses in Weapons of Math Destruction: What happens when these algorithms go wrong? She defines WMDs by three qualities: opacity, scale and damage. Opacity refers to the secrecy surrounding the algorithm and underlying data, Scale is widespread use, and damage are the consequences when the algorithm goes wrong. She covers a wide array of examples in clear, nontechnical language.

One reason to use algorithms in place of human judgement is that humans have a well-established reputation for bias. A common misconception is that algorithms, because they are mathematical are free of bias. O’Neil points out that algorithms reflect the biases (and ignorance) of their creators and the limitations of the underlying data. Perhaps the most striking example here is sentencing algorithms, which attempt to replace biased human judgement with impartial mathematics. In practice, these algorithms reproduce the same racial biases because the data that feeds them–arrest records, zip codes, etc, are themselves full of racial bias.

O’Neil also provides an excellent analysis of the effects of algorithms on our public discourse, where they enable microtargeting: delivering different messages to different potential voters based on detailed electronic dossiers of each. This tool is deliberately opaque, allowing campaigns to “pinpoint vulnerable voters and target them with fear-mongering campaigns… At the same time, they can keep those ads away from the eyes of voters likely to be turned off (or even disgusted) by such messaging”.

Algorithms aren’t going anywhere. We are steaming full speed towards a future where machines increasingly supplement and even supplant human judgement in vast areas of our lives, from hiring decisions to driving. This era is full of both promise and peril. Thus, it is essential understand the dangers of weapons of math destruction and how we can protect ourselves from them. O’Neil is remarkably successful in addressing both of these questions and she manages to do so without resorting to technical language. This book is essentially the algorithm analog to Daniel Kahneman’s excellent catalog of the failures of human judgement, Thinking Fast and Slow. Weapons of Math Destruction is essential reading for anyone living in the modern era, but especially scientists seeking to apply their mathematical tools outside of their discipline.

Find it on: Goodreads, or Amazon