Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Uncertainty regarding rules of AI as well as future development makes it vital to capitalize on emerging technologies to minimize escalation, otherwise risking unmanned catastrophe through NC3 automation (this one is prob more for goldilocks affs than regs)

Horowitz and Scharre 19 (Michael C. Horowitz is Director of Perry World House and Richard Perry Professor at the University of Pennsylvania. He is currently on a leave of absence from the University to serve as Director of the Office of Emerging Capabilities Policy at the U.S. Department of Defense. Paul Scharre is the Vice President and Director of Studies at CNAS. He led the Department of Defense (DoD) working group that drafted DoD Directive 3000.09, establishing the department’s policies on autonomy in weapon systems) December 2019 “A Stable Nuclear Future? The Impact of Autonomous Systems and Artificial Intelligence” https://arxiv.org/ftp/arxiv/papers/1912/1912.05291.pdf) // ZX

Nuclear weapons are arguably the single most significant weapon system invented in modern history, meaning uncertainty about the viability of nuclear deterrence in the 21st century constitutes one of the most important security risks facing the world.2 This uncertainty is both a product and source of increased tensions in nuclear dyads worldwide. The proliferation of conventional military technologies, such as hypersonic weapons, could further undermine deterrence by potentially undermining traditional modes of escalation management, and as a consequence, nuclear stability. 3 The impact of autonomous systems and artificial intelligence (AI) for nuclear stability remains understudied, however.4 In early 2017, Klaus Schwab of the World Economic Forum argued that the world is on the cusp of a Fourth Industrial Revolution, wherein several technologies – but most prominently AI – could reshape global affairs.5 Many defense experts around the world share Schwab’s recognition of the potentially transformative effects of AI.6 The most prominent statements about the impact of AI on warfare, however, tend to be extreme. Elon Musk, for instance, has vocally contended that AI run amok could risk World War III.7 This overheated rhetoric masks the way that advances in automation, autonomous systems, and AI may actually influence warfare, especially in the vital areas of nuclear deterrence and warfighting. The intersection of nuclear stability and artificial intelligence thus raises critical issues for the study of international politics. Relative peace between nuclear-armed states in the 20th century arguably relied in part on mutually assured destruction (MAD). 8 MAD prevails when each side recognizes that both it and its opponent have an assured nuclear second-strike capability, or that either side can impose unacceptable damage on the other in retaliation against a nuclear attack.9 Threat of mutual destruction ultimately led both the United States and the Soviet Union to deprioritize the role of preemption in their nuclear war plans.10 Furthermore, as Albert Wohlstetter found, the threat of mutual destruction “offer[ed] every inducement to both powers to reduce the chance of accidental war.”11 While there are no known instances of accidental war, there are historical examples of unintended escalation, either in preconflict crises or once a conflict is underway.12 Accidental escalation is when a state unintentionally commits an escalatory act (i.e. due to technical malfunction, human error, or incomplete control over military forces). 13 Inadvertent escalation can also occur, whereby a state unknowingly commits an escalatory act (i.e., an intentional act that unknowingly crossing an adversary’s red line). 14 Accidents have increased tensions between countries on numerous occasions, but have not led to escalation.15 Nuclear-armed states have expended vast resources to minimize the risk of unintentional escalation, knowing that it could lead to catastrophe should it occur. Automation may complicate the risks of escalation, deliberate or unintended, in a number of ways. Automation has improved safety and reliability in other settings, from nuclear power plants to commercial airliners. Used properly, many applications of automation in nuclear operations could increase reliability, reduce the risk of accidents, and buy more time for decision-makers in a crisis. Automation can help ensure that information is quickly processed, national leaders’ desires are swiftly and efficiently conveyed, and launch orders are faithfully executed. On the other hand, poor applications of automation could render nuclear early warning or command-and-control (C2) systems more opaque to users, leading to human-machine interaction failures. Human users could fall victim to automation bias, for example, surrendering their judgment to the system in a crisis Automation is often brittle and lacks the flexibility humans have to react to events in their broader context. The states most likely to be willing to tolerate these risks for the perceived capability gains would be those that have significant concerns about the viability of their second-strike deterrents. Thus, the more a country fears that, in a world without using autonomous systems, its ability to retaliate to a nuclear strike would be at risk, the more attractive autonomous systems may appear. Uninhabited nuclear delivery platforms could undermine nuclear surety, as they could be hacked or slip out of control, potentially leading to accidental or inadvertent escalation. Automated systems could end up reducing decision-maker flexibility by narrowing options, hampering attempts to manage escalation. These dynamics suggest that autonomous systems could influence the potential for nuclear escalation in three ways. First, while many aspects of the nuclear enterprise are already automated in many countries, from early warning and command and control to missile targeting, as autonomous systems improve, states may elect to automate new portions of the early warning and C2 processes to improve both performance and security. From a security standpoint, for instance, increased automation in nuclear early warning may allow operators to identify threats more rapidly in a complex environment. Likewise, automation may help to ensure the dissemination of launch orders in a timely manner in a degraded communications environment. States may also automate – or threaten to automate – nuclear launch procedures in the belief that doing so would provide them with a coercive advantage over adversaries. Second, as military robotics advance, nuclear powers could deploy uninhabited nuclear delivery platforms for a variety of reasons. For instance, a state might deploy nuclear-armed long endurance uninhabited aerial vehicles (UAVs) in the belief that doing so would provide additional nuclear signaling or strike options. They might also look to uninhabited nuclear delivery platforms to bolster their secure second-strike capabilities. Nuclear delivery vehicles – such as torpedoes – capable of autonomously countering enemy defenses or selecting targets might be seen to do likewise. Alternatively, a government might choose to automate its nuclear forces so that a small number of trusted agents can maintain control. This might could be especially attractive for a nuclear-armed regime that fears a coup or other forms of interference by its nation’s military elite. Third, the increased automation of conventional military systems might influence nuclear stability in direct and indirect ways.16 It may enable – or more likely yet, be seen to enable – improved counterforce operations by technologically-advanced states. The ineffectiveness of counterforce operations – and hence the survivability of second-strike deterrents – presently hinges in large part on the difficulty of finding and tracking adversary nuclear launch platforms (mobile missiles or submarines) long enough for ordnance to be delivered. Machine learning algorithms and other applications of artificial intelligence could, in principle, improve states’ abilities to collect and sift through large amounts of data in order to locate and track such targets, though it is important to recognize limitations to any developments given the real-time requirements for a disarming strike. Likewise, military autonomy could enable the deployment of conventional autonomous systems designed to shadow and/or attack nuclear-armed submarines. Furthermore, if automation gives (or is perceived to give) one side in a competitive dyad a significant conventional military advantage, the weaker side may feel compelled to rely more heavily on nuclear weapons for deterrence and warfighting. These issues surrounding the potential impacts of artificial intelligence are magnified by uncertainty about the trajectory of technological developments. This article first proceeds by clarifying what autonomous systems are and clarifying often-tricky definitional issues surrounding artificial intelligence. It then lays out some key theoretical expectations. Second, the article explores the impact of autonomous systems on early warning and nuclear command and control, as well as intelligence, surveillance, and reconnaissance (ISR) relevant for nuclear systems, in the context of recent research. Third, the article discusses the potential for uninhabited nuclear delivery platforms and vehicles featuring new kinds of automation. Fourth, the article describes the way conventional autonomous systems could both directly and indirectly influence nuclear stability. Finally, the article concludes by assessing the net likely impact of autonomous systems on nuclear stability and describing potential pathways for future research. The analysis argues that the impact of autonomous systems could depend on the specific application – both where automation falls in the nuclear enterprise but also how it is implemented in terms of design, human-machine interfaces, training, and operator culture.

Sign in to recut, update, or save this card.