A former Wall Street
quant sounds an alarm on the mathematical models that pervade modern
life — and threaten to rip apart our social fabric
We live in the age
of the algorithm. Increasingly, the decisions that affect our
lives—where we go to school, whether we get a car loan, how much we
pay for health insurance—are being made not by humans, but by
mathematical models. In theory, this should lead to greater fairness:
Everyone is judged according to the same rules, and bias is
eliminated.
But as Cathy O’Neil
reveals in this urgent and necessary book, the opposite is true. The
models being used today are opaque, unregulated, and uncontestable,
even when they’re wrong. Most troubling, they reinforce
discrimination: If a poor student can’t get a loan because a
lending model deems him too risky (by virtue of his zip code), he’s
then cut off from the kind of education that could pull him out of
poverty, and a vicious spiral ensues. Models are propping up the
lucky and punishing the downtrodden, creating a “toxic cocktail for
democracy.” Welcome to the dark side of Big Data.
Tracing the arc of a
person’s life, O’Neil exposes the black box models that shape our
future, both as individuals and as a society. These “weapons of
math destruction” score teachers and students, sort résumés,
grant (or deny) loans, evaluate workers, target voters, set parole,
and monitor our health.
O’Neil calls on
modelers to take more responsibility for their algorithms and on
policy makers to regulate their use. But in the end, it’s up to us
to become more savvy about the models that govern our lives. This
important book empowers us to ask the tough questions, uncover the
truth, and demand change.