In 2015, Google Photos labelled a photo of two Black people as "gorillas". It was not an isolated error, nor the result of racist programmers: it was the predictable consequence of an image recognition system trained on a dataset that under-represented Black people. The system had "learned" that certain pixel patterns corresponded to "gorilla" because in the training data those patterns were not sufficiently associated with "person".
This incident — one of many — illustrates a fundamental truth: algorithmic bias does not originate in the code, it originates in the data. And data is the product of the society that generates it. If a society discriminates on the basis of race, gender or social class, the data it produces will reflect those discriminations. An algorithm trained on that data will only encode and amplify the same inequalities.
But bias in data is not the only source. Safiya Umoja Noble, in her book "Algorithms of Oppression", documents how design choices — how we classify information, which metrics we optimise, how we define "success" — incorporate values and priorities that are never neutral. If a hiring system is optimised to find candidates "similar" to current employees, and the company is predominantly white and male, the system will perpetuate that composition. Discrimination is not a bug: it is the system working exactly as designed.
There are at least three levels of bias we must recognise. Historical bias: data reflects past discrimination (e.g. credit history reflects decades of discrimination in credit access for minorities). Representation bias: some groups are under- or over-represented in datasets. Measurement bias: the variables we measure do not capture reality fairly (e.g. using postcode as a proxy for "risk", knowing that postcodes correlate with race due to residential segregation).
Recognising these sources of bias is essential because it protects us from a false solution: believing that "collecting more data" or "improving the algorithm" is enough to eliminate discrimination. The problem is not only technical — it is social, historical, political. And it requires solutions that operate on all these levels.