Use of AI leads to racial bias and sexism.
Ulrike Franke, 2021 (Senior Policy Fellow, European Council on Foreign Relations), ARTIFICIAL DIVIDE: HOW EUROPE AND AMERICA COULD CLASH OVER AI. Jan. 2021. Retrieved May 18, 2022 from https://ecfr.eu/publication/artificial-divide-how-europe-and-america-could-clash-over-ai/
Machine-learning systems are those that use computing power to execute algorithms that learn from data. This means that AI is only as good as the algorithm it uses and the data it is being trained on. If, for example, the data is incomplete or biased, the AI trained on it will be equally biased. AI researchers around the world, and especially researchers from minority groups, have raised the alarm about this particular risk, which has already materialised in several cases. In the US, a risk assessment tool used in Florida's criminal justice system labelled African-American defendants as "high risk" at nearly twice the rate as white defendants. A hiring algorithm used at Amazon penalised applicants from women's colleges, while a chatbot trained on Twitter interactions started to post racist tweets. The concern is that real-life data fed into machine-learning systems perpetuate existing human biases, and that - as humans tend to consider computers to be rational - these biases will effectively be sanctioned, thereby entrenching prejudice further in society. Furthermore, AI trained on datasets collected in one cultural context and deployed in another cultural context might effectively enable cultural imperialism. In response to these concerns, big tech firms have developed principles and guidelines, and created research groups and divisions, on ethical AI. More recently, however, scandals have emerged over big tech employees reportedly being forced to leave their jobs for being too critical, heightening concerns that these companies are not taking the issue seriously enough.