Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

When AI is fed real life data it inherently perpetuates prejudices already found in society

Franke ’21, (Ulrike Esther Franke, senior policy fellow at the European Council on Foreign Relations, “ARTIFICIAL DIVIDE: HOW EUROPE AND AMERICA COULD CLASH OVER AI,” ECRF, January 2021, https://ecfr.eu/wp-content/uploads/Artificial-divide-How-Europe-and-America-could-clash-over-AI.pdf)

Machine-learning systems are those that use computing power to execute algorithms that learn from data. This means that AI is only as good as the algorithm it uses and the data it is being trained on. If, for example, the data is incomplete or biased, the AI trained on it will be equally biased. AI researchers around the world, and especially researchers from minority groups, have raised the alarm about this particular risk, which has already materialised in several cases. In the US, a risk assessment tool used in Florida’s criminal justice system labelled African-American defendants as “high risk” at nearly twice the rate as white defendants. A hiring algorithm used at Amazon penalised applicants from women’s colleges, while a chatbot trained on Twitter interactions started to post racist tweets. The concern is that real-life data fed into machine-learning systems perpetuate existing human biases, and that – as humans tend to consider computers to be rational – these biases will effectively be sanctioned, thereby entrenching prejudice further in society. Furthermore, AI trained on datasets collected in one cultural context and deployed in another cultural context might effectively enable cultural imperialism. In response to these concerns, big tech firms have developed principles and guidelines, and created research groups and divisions, on ethical AI. More recently, however, scandals Artificial divide: How Europe and America could clash over AI – ECFR/367 4 have emerged over big tech employees reportedly being forced to leave their jobs for being too critical, heightening concerns that these companies are not taking the issue seriously enough. Related to concerns about bias are those about the transparency of how AI works. Employing machine-learning methods means that systems are no longer programmed – namely, told what to do by human beings – but instead learn how to behave either by themselves or under human supervision. It is difficult for a human to understand and track how an AI-enabled system has reached a conclusion. This makes it hard to challenge AI-enabled decisions, and to tell whether malicious actors have exploited the vulnerabilities of AI systems. Problematic context Even if AI-enabled systems were proven to be perfectly reliable and unbiased, there are contexts in which delegating decisions to machines may be inherently problematic. This includes using AIenabled systems to make decisions that have fundamental implications for an individual’s life, such as in a judicial or military context. In the military context, lethal autonomous weapon systems able to exert force without meaningful human control or supervision are particularly controversial. The concern is a moral one: should a machine – no matter how intelligent – be allowed to make decisions about the physical wellbeing, or indeed life and death, of a human being? The European Parliament answered this question in the negative, passing a resolution in 2018 that urged the EU and its member states “to work towards the start of international negotiations on a legally binding instrument prohibiting” such weapons.

Sign in to recut, update, or save this card.