Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

The Pressure to deploy AI Rapidly makes accidents inevitable – the technology is brittle

Arnold and Toner, 2021 – Center for Security and Emerging Threats [Zachary and Helen, July, CSET Policy Brief. “AI Accidents: An Emerging Threat What Could Happen and What to Do” https://cset. georgetown.edu/wp-content/uploads/CSET-AI-Accidents-An-Emerging-Threat.pdf Acc 6/7/22 TA]

Despite these problems, AI systems are becoming integrated into the real world at a pace that is only expected to accelerate in the next decade.9 These systems may be fragile, but as companies, governments, and militaries decide when and how to deploy them, their huge potential benefits will often overshadow uncertain risks. Leaders in these organizations also may not be fully aware of these risks, and may face pressure from competitors willing to move quickly.10 To be sure, some industries are already deploying AI much faster than others, and a few sensitive sectors may remain “walled off” for some time.11 But eventually, the powerful incentives driving the spread of AI today are likely to make it pervasive. As our economy, security, and health become more and more dependent on AI systems, these systems’ fragilities will put lives at stake. Today, many are worried about AI being misused intentionally. An adversary could attack with swarms of drones; authoritarian governments are already using AI algorithms to discriminate on the basis of race or ideology. These risks are real, and they deserve attention. But unintended, accidental AI disasters are also an urgent concern. AI-related accidents are already making headlines, from inaccurate facial recognition systems causing false arrests to unexpected racial and gender discrimination by machine learning software.12 This is especially striking since AI has so far mostly been deployed in seemingly lower-stakes settings, such as newsfeed rankings, ad targeting, and speech recognition, with less deployment in higher stakes areas such as autonomous driving. Despite these initial accidents, governments, businesses, and militaries are preparing to use today’s flawed, fragile AI technologies in critical systems around the world. Future versions of AI technology may be less accident-prone, but there is no guarantee—and regardless, if rollout continues as expected, prior versions of the technology may already have been deployed at massive scale. The machine learning models of 2020 could easily still be in use decades in the future, just as airlines, stock exchanges, and federal agencies still rely today on COBOL, a programming language first deployed in 1960.13 In retrospect, even the most extreme technological accidents, from the Challenger disaster to the meltdown at Chernobyl, can seem both predictable and preventable.14 History is full of accidents that seem obvious in retrospect, but “no one could have seen coming” at the time. In other cases, known risks are brushed aside, or obvious fixes go unmade. Unless we act, there is no reason to think that the advent of AI will be any different. In fact, there are reasons to think AI could cause more accidents than other technologies that have caused high-profile disasters. Unlike the space shuttle or nuclear power plants, for example, AI will be pervasive throughout society, creating endless opportunities for things to go awry. What’s more, modern AI is so good at some tasks that even sophisticated users and developers can come to trust it implicitly.15 This degree of trust, placed in pervasive, fallible systems without any common sense, could have terrible consequences.

Sign in to recut, update, or save this card.