What you need to know about algorithmic bias

Algorithmic bias is real and harmful. But what is algorithmic bias, and what can we do about it?

What is algorithmic bias?

First, what is machine learning bias? Hardt (2017) defines a biased model as one that exhibits systematically adverse discrimination based on “socially salient qualities that have served as the basis for unjustified and systematically adverse treatment in the past.” Algorithms are meant to discriminate; all models have a worldview, and in that sense exhibit bias. When we discuss algorithmic bias, we are concerned with those socially salient qualities that have served as the basis for unjustified and systematically adverse treatment in the past (e.g., race, gender, disability, income, class, and so on).

That's what algorithmic bias is. What can we do about it?

Here's what we can do about algorithmic bias.

1. Leave the socially  salient features in

If we want to know where an algorithm exhibits bias, we must have access to those socially salient features—even if they seem irrelevant to the problem. For example, in our work on healthcare , a company that built a medical risk-scoring algorithm did not include race. Why would you? A patient's race shouldn't factor into the quality of healthcare treatment they receive... right? Indeed, it shouldn't. But only if we have access to data about race can we say with confidence that the algorithm exhibits no systematically adverse discrimination across racial groups.

Spoiler alert: this healthcare risk-scoring algorithm did exhibit racial bias; the algorithm undervalued the risk of Black patients because some of the features the model used turned out to be correlated to race—and, since race wasn't included in the model, the model's designers and users didn't notice until the model was already exposed to hundreds of millions of patients.

We must always have access to socially salient features. Any notion of an audit will be impossible without access to such features.

2. Machine learning bias cannot be "fixed," only made less harmful

Algorithmic bias is “sociotechnical”—it is not caused by technical problems alone, and cannot be “solved” by technical solutions alone. Technical approaches can help identify and (to a point) ameliorate bias. However, bias is both the cause and effect of social issues. Biased algorithms arise from social bias in data (e.g., not enough enough Black faces in a dataset), biased problem framing (e.g., attempting to predict criminality based on someone's face); occasionally, complex models will invent new sources of bias on their own. Biased models go on to perpetuate harms, which cause their own social problems. And so the loop continues.

There is no silver-bullet engineering fix for bias. Anyone telling you they have one is likely trying to sell you something. There are tools for identifying and ameliorating harmful outcomes, but bias isn't fixable (see #2). Ameliorating bias requires hard and consistent work over time. Algorithms must be regularly checked for bias as part of the engineering process.  

No silver-bullet engineering solution will "fix" or "remove" bias. Algorithmic bias can, at best, be made less harmful, and non-illegal.

3. Identifying and ameliorating bias is highly context-dependent

Because algorithmic bias is fundamentally about socially salient qualities, any approach to ameliorate it is fundamentally contextual. Understand the true impact of harms—the historical antecedents for the harms that algorithms perpetuate. Only with that understanding can bias be addressed.

Understand the true impact of harms—the historical antecedents for the harms that algorithms perpetuate.

Get in touch

Algorithmic bias may not be fixable, but you can limit its harm. We can help you train your team to handle bias more effectively or, as needed, reach out to the right auditing firm for your job. Get in touch to schedule a free consultation.