Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

An AI Arms Race will increase nuclear instability – nations will rely on AI command and control, which is vulnerable to disinformation and interference

Fitzpatrick, 2019 - former the IISS Non-Proliferation and Nuclear Policy Programme [Mark, 21 May, “Artificial Intelligence and Nuclear Command and Control,” https://www.iiss.org/blogs/survival-blog/2019/04/artificial-intelligence-nuclear-strategic-stability 6/18/22 MD]

The year was 2021. The fictionalised events occurred in an IISS tabletop exercise in London in November 2018, for a project funded by the Carnegie Corporation of New York designed to examine the potential impact of artificial intelligence (AI) on nuclear strategic stability. Experts from China, Russia, the United States, the United Kingdom and the Czech Republic played roles as decision-makers in Beijing, Moscow and Washington as they sought to understand, address and take advantage of the scenarios and variations posed by the IISS control team. The major powers’ nuclear command-and-control systems increasingly rely on AI programmes, or, more precisely, expert systems and machine-learning algorithms, to enhance information flow, situational awareness and cyber security. Such capabilities can provide such systems with a larger window of opportunity in which to respond in the event of a crisis and thereby support de-escalation. Malevolent actors, however, can also use new technologies offensively to deceive, disrupt or impair command-and-control systems and their human controllers. The IISS tabletop exercise sought to explore several of these malfeasance pathways and vulnerabilities in plausible crisis scenarios. It demonstrated how an AI arms race could reduce strategic stability as the nuclear-weapons states become more reliant on AI for strategic warning in relation to nuclear command-and-control functions. Escalation via ‘deep fakes’ A key danger played out in the exercise was the potential for third parties to spoof warning systems and embed disinformation to fool human operators. In the scenario, a hitherto unknown non-state actor, the World Peace Guardians, circulated falsified videos and photographs on social-media platforms to create the impression that three US special-operations-forces soldiers had been killed by nerve gas in clashes with Russian military advisers in Syria. Some US pundits argued the legal case for the use of tactical nuclear weapons in response. Doctored videos then appeared on American and Chinese media platforms that showed the families of several prominent US officials hurriedly departing Washington DC. Further eyewitness accounts on social media claimed that missile silos in the western US had gone to high alert, and that the crews of two had even opened the silo doors. As a result of these and other secondary indicators used by Russian AI-driven situational-awareness algorithms, Moscow’s strategic-warning systems began informing the Kremlin leadership that a US missile launch could be imminent. In the first situation report generated by the control team, Chinese President Xi Jinping cautioned the US not to conduct a nuclear strike against any country and warned that if the US did not provide ‘credible evidence’ that it was not mobilising for war, China would have to take unspecified defensive actions. Generally speaking, the scenario was a plausible example of escalation by third parties. In this case, a non-state actor made a deep fake sufficiently believable that it generated a crisis between two nuclear states. A workshop background paper explained how offensive AI capabilities could widen the psychological distance between the attacker and its target. The paper fore-warned that a third-party actor might attempt to use AI-driven adversarial inputs, data-poisoning attacks, and audio and video manipulation to create escalatory effects between nations. While other early-warning systems would eventually discredit the spoof, it would still create high levels of uncertainty and tension in a short period of time. Doctored videos would likely force both parties to put their respective militaries on high alert. They would utilise overhead imagery, signals intelligence or human reporting to determine the reality on the ground. Collecting and processing the intelligence would take precious time during an escalatory crisis in which AI algorithms were urging immediate response actions.

Sign in to recut, update, or save this card.