Free beta access. 50 free AI tokens each week. Optional top-ups available. See what is included
← SearchMichigan2022Disadvantage
Opening Word document…

NATO key to military development and standardization

Stanley-Lockman and Trabucco ’22 -- (Zoe Stanley-Lockman and Lena Trabucco, 2022, "NATO’s Role in Responsible AI Governance in Military Affairs," https://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780197579329.001.0001/oxfordhb-9780197579329-e-69, accessed 6-23-2022) -- nikki

Safety and security For humans to meet ethical and legal commitments when developing and deploying AI, the systems themselves must be safe, secure, and reliable. More simply put, if humans and institutions interacting with AI do not have confidence that the systems will perform as expected, then they cannot assure that its development and deployment are responsible. This makes safety and security a key pillar of responsible AI governance for any actor.82 As this section explores for NATO in particular, safety and security are indispensable to the Alliance’s stated goals to focus its approach to EDTs in the areas of “deterrence and defense, capability development, legal and ethical norms, and arms control aspects.”83 Politically, democratic militaries using AI cannot be accountable to their citizenries nor their coalition partners if they lack mechanisms to trace and explain how their systems are reliable. Accidents and interference with AI systems could likewise create political risks for the Alliance. For example, if deepfakes and micro-targeted information attacks compromise confidence in the integrity of information used to build a common operating picture, then the operational difficulties could also erode political trust between Allies in a few key ways. In the North Atlantic Council, disagreement about the integrity of information could slow the decision-making body’s ability to react to fast-changing operational realities.84 Further, compromised AI systems may not only make it harder for forces to prevent harm to non-combatants, but also to prevent friendly fire. In this way, coalition forces arguably face even higher obligations to coordinate on the reliability of their systems, relative to adversaries and near-peer competitors that tend to operate alone. As such, responsible AI governance is not purely technical; policy alignment and strategic planning are likewise necessary to draw attention to risk management above the tactical level. Even without being attacked, governability of AI in a NATO context also means understanding how AI-enabled and autonomous systems developed by the 30 Allies—and other partners—will interact with one another. NATO has expressed interest in governability as a principle of AI “to disengage or deactivate in case of unintended behavior,”85 which echoes the U.S. Department of Defense definition of governable AI.86 Disengaging adversaries is important to maintain de-escalation measures in conflict. For NATO, interoperability between systems also relates to governable AI because allies must also consider how the interactions between the 30 Allies’ own AI-enabled and autonomous systems may result in unintended or emergent behavior.87 This means that NATO has a responsibility to coordinate activities—be they technical exchanges, standardization efforts, or training and exercises—to build confidence that the systems perform as humans intend.88 Without this coordination, the lack of interoperability of allied systems could lead to accidents, and separately, the potential loss of operational effectiveness also presents vulnerabilities for adversaries to exploit. In addition to governability, NATO and its Allies are assessing the risks that bias, attacks, and lack of interpretability can introduce in relation to the anticipated uses of a given AI system.89 In security and defense, new and heightened risks include poisoning of the information environment, deception systems and techniques, uncertainty about the performance of systems in new and unknown environments, and the possibility that tensions or accidents escalate at a faster tempo than humans and institutions can process, among others. These risks can manifest either in motivated attacks or unintentional failure modes.90 In both cases, assuring and certifying that military assets are safe and secure is important given the inherently high risk in operational environments. These operational environments include the presumption that an adversary is disrupting one’s own systems, be it by directly attacking the AI systems themselves, or disrupting the broader command, control, and communications systems under which the AI systems are operating.91 Mitigating these types of risks is typically done in testing, evaluation, validation and verification (TEVV) and in experimentation activities.92 Yet AI cannot be validated and verified the way traditional software systems are because there is no guarantee that an AI system will perform in the real world as it does in a testing environment, and because lifelong-learning systems will perform differently over their lifecycle. Having robust assurance and TEVV processes in place are also important for operators to build trust in the systems they are meant to use, as well as for citizenries and coalition partners at large to see that accountability procedures still apply. As such, building institutional procedures to govern AI safety and security is necessary to build trust in the use of the technology—as well as to develop countermeasures and defensive systems that protect against adversarial threats. NATO thus has an institutional responsibility to prevent and mitigate these intentional and unintentional failures if using AI in operations and mission support.93 As Table 69.1 shows, the Alliance also has a range of relevant entities to coordinate national approaches to AI safety and security, as well as facilitate safety measures as part of responsible use in the Alliance-wide ecosystem. NATO has an important role to play in military standardization and Allied policy planning for safe, secure, and interoperable AI. This includes the coordinating role of the Conference of National Armaments Directors and the Command, Control and Consultation Board to implement complementary acquisition processes that fuse AI adoption measures with safety responsibilities. Furthermore, entities including STO and NSO have a significant role setting the technical baseline and promulgating materiel standards that provide the technical framework for safety and security. Although their staffs are themselves small, they both convene hundreds, if not thousands, of subject matter experts in working groups. As such they both offer unique technical networks to help shape safety and security in a way that minimize risk in operations. NATO’s resources and leadership are vital to using standards and coalition policy to instill safe and secure technological development, a necessary condition to interoperable and successful future operations.

Sign in to recut, update, or save this card.