Free beta access. 50 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Uncoordinated adoption of AI systems undermines NATO interoperability, making miscalculation and accidents more likely.

Trabucco and Stanley-Lockman, 2022 – prof of Political Science, University of Copenhagen and prof of Defense and Strategic Studies, Nanyang Technological University [Lena and Zoe, The Oxford Handbook of AI Governance, March, “NATO’s Role in Responsible AI Governance in Military Affairs” https://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780197579329.001.0001/oxfordhb-9780197579329-e-69 Acc 4/16/22 TA]

Even without being attacked, governability of AI in a NATO context also means understanding how AI-enabled and autonomous systems developed by the 30 Allies—and other partners—will interact with one another. NATO has expressed interest in governability as a principle of AI “to disengage or deactivate in case of unintended behavior,”85 which echoes the U.S. Department of Defense definition of governable AI.86 Disengaging adversaries is important to maintain de-escalation measures in conflict. For NATO, interoperability between systems also relates to governable AI because allies must also consider how the interactions between the 30 Allies’ own AI-enabled and autonomous systems may result in unintended or emergent behavior.87 This means that NATO has a responsibility to coordinate activities—be they technical exchanges, standardization efforts, or training and exercises—to build confidence that the systems perform as humans intend.88 Without this coordination, the lack of interoperability of allied systems could lead to accidents, and separately, the potential loss of operational effectiveness also presents vulnerabilities for adversaries to exploit. In addition to governability, NATO and its Allies are assessing the risks that bias, attacks, and lack of interpretability can introduce in relation to the anticipated uses of a given AI system.89 In security and defense, new and heightened risks include poisoning of the information environment, deception systems and techniques, uncertainty about the performance of systems in new and unknown environments, and the possibility that tensions or accidents escalate at a faster tempo than humans and institutions can process, among others. These risks can manifest either in motivated attacks or unintentional failure modes.90 In both cases, assuring and certifying that military assets are safe and secure is important given the inherently high risk in operational environments. These operational environments include the presumption that an adversary is disrupting one’s own systems, be it by directly attacking the AI systems themselves, or disrupting the broader command, control, and communications systems under which the AI systems are operating.91 Mitigating these types of risks is typically done in testing, evaluation, validation and verification (TEVV) and in experimentation activities.92 Yet AI cannot be validated and verified the way traditional software systems are because there is no guarantee that an AI system will perform in the real world as it does in a testing environment, and because lifelong-learning systems will perform differently over their lifecycle. Having robust assurance and TEVV processes in place are also important for operators to build trust in the systems they are meant to use, as well as for citizenries and coalition partners at large to see that accountability procedures still apply. As such, building institutional procedures to govern AI safety and security is necessary to build trust in the use of the technology—as well as to develop countermeasures and defensive systems that protect against adversarial threats. NATO thus has an institutional responsibility to prevent and mitigate these intentional and unintentional failures if using AI in operations and mission support.93 As Table 69.1 shows, the Alliance also has a range of relevant entities to coordinate national approaches to AI safety and security, as well as facilitate safety measures as part of responsible use in the Alliance-wide ecosystem.

Sign in to recut, update, or save this card.