Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

AI is racist – healthcare proves

Mohamed et al. 20 (Shakir Mohamed is Senior Staff Scientist at DeepMind, Marie Therese, William Isaac, 07-12-20, “Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence” https://link.springer.com/content/pdf/10.1007/s13347-020-00405- pp. 662)-qcl

The limitations of these value principles become clearer as AI and other advanced technologies become enmeshed within high-stakes spheres of our society. Initial attempts to codify ethical guidelines for AI, e.g. the Asilomar principles (Asilomar Meeting 2017), focused on risks related to lethal autonomous weapons systems and AGI Safety. Though both are critical issues, these guidelines did not recognise that risks in peace and security are first felt by conflict zones in developing countries (Garcia 2019), or engage in a disambiguation of social safety and technical safety. Moreover, they did not contend with the intersection of values and power, whose values are being represented, and the structural inequities that result in an unequal spread of benefits and risk within and across societies. An example of this nexus between values, power and AI is a recent study by Obermeyer et al. (2019), which revealed that a widely used prediction algorithm for selecting entry into healthcare programs was exhibiting racial bias against AfricanAmerican patients. The tool was designed to identify patients suitable for enrolment into a “high-risk care management” programme that provides access to enhanced medical resources and support. Unfortunately, large health systems in the USA have emphasised contextual values to “reduce overall costs for the healthcare system Decolonial AI: Decolonial Theory as Sociotechnical 661 while increasing value” (AMA 2018) or “value for money” (UK National Health Service 2019) on “value for money”) when selecting potential vendors for algorithmic screening tools at the expense of other values such as addressing inequities in the health system. As a result, the deployed algorithm relied on the predictive utility of an individual’s health expenses (defined as total healthcare expenditure) indirectly leading to the rejection of African-American patients at a higher rate relative to white patients, denying care to patients in need, and exacerbating structural inequities in the US healthcare system (Nelson 2002). As this example shows, the unique manner in which AI algorithms can quickly ingest, perpetuate and legitimise forms of bias and harm represents a step change from previous technologies, warranting prompt reappraisal of these tools to ensure ethical and socially beneficial use. An additional challenge is that AI can obscure asymmetrical power relations in ways that make it difficult for advocates and concerned developers to meaningfully address during development. As Benjamin (2019) notes, “whereas in a previous era, the intention to deepen racial inequities was more explicit, today coded inequity is perpetuated precisely because those who design and adopt such tools are not thinking carefully about systemic racism”. Some scholars such as Floridi et al. (2018) have highlighted that technologies such as AI require an expansion of ethical frameworks, such as the Belmont Principles, to include explicability (explanation and transparency) or non-malfeasance (do no harm). Whittlestone et al. (2019) conversely argue for a move away from enumerating new value criteria, and instead highlight the need to engage more deeply with the tensions that arise between principles and their implementation in practice. Similarly, we argue that the field of AI would benefit from dynamic and robust foresight tactics and methodologies grounded in the critical sciences to better identify limitations of a given technology and their prospective ethical and social harms.

Sign in to recut, update, or save this card.