Many would take this thought experiment as an example of algorithmic unfairness. It does not only describe how algorithms themselves can result in unfair results, but also how algorithms can reinforce already existing economic and societal bias.

Fairness and bias are probably the most discussed ethical issues related to the contemporary algorithms. Why are they so central?

• Firstly, fairness is a fundamental element of social stability. As the philosopher John Rawls remarks, the stability of a society – or any group – depends upon the extent to which the members of that society feel that they are being treated in a just manner. When some of society's members feel that they are treated in an unfair manner, it usually creates a foundation for social unrest, disturbances, and strife. People hold social unity only to the extent that their institutions are fair.
• Secondly, as Immanuel Kant remarked, human beings have the same dignity. In virtue of this dignity they are entitled to be treated as equals. If individuals are treated unfairly – especially on arbitrary grounds – their fundamental human dignity is violated. When this violation is implemented in practices, it leads to discrimination.

However, as the example of grade inflation algorithm illustrates, fairness is a complex issue. The algorithm was designed to correct grade inflation because it was thought to be unfair if students got an unfair advantage on their grades. As a result, paradoxically, the algorithm ended up reinforcing already existing societal bias.

In this chapter, we’ll focus on fairness, biases and discrimination. We’ll address questions such as: What, exactly, is fairness? Should fairness consist of ensuring everyone has an equal probability of obtaining some benefit? Or should fairness take the individual differences into account, and recognize the diversity? And finally, are fairness and discrimination synonymous, or do they mean separate things?