Free beta access. 50 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

NATO has unique authority over AI – legitimacy, legal interoperability, and accountability.

Lockman and Trabucco 22 Zoe Stanley-Lockman (Defense and Strategic Studies, Nanyang Technological University) and Lena Trabucco (Political Science, University of Copenhagen), 3-20-2022, "NATO’s Role in Responsible AI Governance in Military Affairs," The Oxford Handbook of AI Governance, https://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780197579329.001.0001/oxfordhb-9780197579329-e-69#oxfordhb-9780197579329-e-69-div1-4//mb24

The first pillar considers NATO’s role in the evolution of ethical and values-driven AI. One ongoing debate regarding AI as a military technology is the ethical implications and baseline values the Allies, and others, want infused in the development and adoption of AI. The Allies themselves lack uniform consensus on numerous, substantial ethical questions on the use of AI, as most clearly seen in the adjacent area of the ethics of autonomy in weapons systems including lethal autonomous weapon systems (LAWS). In this discussion, we spotlight NATO’s role in facilitating and shaping ethical harmonization as an operational requirement to ensure successful future missions. The second pillar examines legal norms as a domain wherein legal uncertainty regarding AI has tangible implications for Allied legal interoperability, a subset of larger coalition interoperability. Thus far, the legal debate regarding AI has been largely fixed on the issue of a treaty banning the use of LAWS. In this section, we advocate for a more nuanced legal picture in which NATO can facilitate legal coordination and tackle some of the foundational legal issues which will prevent successful legal interoperability in future operations. The third pillar identifies safety and security of AI systems as prerequisite to trustworthy and responsible AI in any context, but especially so for the conduct of military activity. At the NATO level, Allied forces must ensure their systems interoperate safely and predictably both to ensure effective command and control (C2) internally, and to prevent disruptions from attacks. It is a foundational facet of coordination that shows the overlap between NATO interests in military effectiveness and incentivization for responsible innovation. Ethics and values One of the vital aspects of AI which has garnered significant global attention is the ethical implications of artificial intelligence as a military technology—an issue that has divided much of the global community, including NATO member states. As a starting point, researchers and analysts have considered the implications of emerging military technology in terms of ethical responsibility and regulation, especially as states and organizations continue to release AI ethical principles, guidelines, and standards.55 We explore how NATO can operationalize the debate around ethics and values of military AI to garner coordination and continue progress of EDT harmonization among partners. Building on the theoretical discussion from STS and military innovation literature above, the adoption of technologies that reinforce values serves the strategic interest of NATO to shape technological innovation against current waves of illiberalism. Additionally, infusing AI development with certain ethical principles and values can have operational advantages and benefits, and NATO can, in particular, promote the ethical principles as operational standards for the Allies. A common critique within the ethics debate is that approaching new technology with an ethical or democratic values-driven perspective translates into comparative military disadvantage. Essentially, if your adversary develops technology without the constraints of ethical principles then there will be diminished effectiveness on the battlefield.56 We find this critique unfounded because it assumes there is a false trade-off between ethics and effectiveness; instead, we argue ethical foundations are built into the architecture of modern warfare.57 As such, ethics is a background condition for battlefield effectiveness, which is already infused in military decision-making and helping to guide the boundaries of international humanitarian law. As such, ethical guidelines do not have to detract from a military’s capacity or competency to devise means and methods of warfare that will serve their national or coalition interest.58 If anything, a first-mover advantage can incentivize an ethical and values-driven AI to establish the threshold of technological standards globally.59 The political dimension of the Alliance rests on the bedrock of a shared commitment to the “principles of democracy, individual liberty and the rule of law,” as enshrined in the foundational North Atlantic Treaty of 1949.60 Shared values are important for NATO operations because they help constitute their legitimacy. In addition to the North Atlantic Council exerting civilian oversight over NATO operations, legitimacy also includes respect for international legal principles including the core principles of international humanitarian law, or the laws of armed conflict, distinction, proportionality, and necessity. Without political oversight and legitimacy, NATO’s military power would be less effective at shaping norms and promoting stability in the international system. The introduction of AI means that NATO has the moral and strategic imperative to adopt technologies that confer legitimacy and responsible innovation.61 Acting on a shared commitment to democratic values is vital to the political cohesion of the NATO Alliance, just as much as it is a determinant of military effectiveness in a predictable security environment. Put simply, shared values are important to both political and operational coherence between Allies. In its 2018 Framework for Future Alliance Operations, the strategic command Allied Command Transformation urged discussion of the legal and ethical dimensions of technological advancement to both know how it would impact NATO decision-making and how the Alliance would be prepared to address adversaries who do not share in that vision.62 As such, NATO is contending with the ways that ethical AI impacts its own cohesion internally and how differences between allies may project outward in the face of competitors whose ethical frameworks and commitment to the rule of law differ. Internally, there is a strong national government commitment to responsible AI. Recently, transatlantic cooperation has initiated partnerships of largely NATO states committed to advancing responsible AI with goals towards data sharing and future interoperability.63 AI defense partnerships are not restricted to military innovation but rather aim to facilitate civilian innovation cooperation for defense purposes. Externally, as AI-enabled autonomous systems enter the arsenals of more technologically advanced countries, uncoordinated ethical frameworks between Allies could pose operational risks. Without wider alignment, AI systems will have “varying technical specifications based on the legal and policy decisions made by individual governments when answering the key questions.”64 Further, although one motivation of autonomous systems is the increased safety of military personnel by removing them from dangerous situations, the lack of alignment could lead some Allies to perceive other capitals’ deployments of unmanned forces as a lack of commitment to put lives on the line, therein posing credibility risks for Allies to assure one another.65 These credibility risks can be mitigated by accountability and verification standards and procedures that NATO can implement for multinational operations, and efforts to institutionalize these procedures for AI are underway.66 While the NATO AI Strategy is expected to create a common foundation for the Alliance’s pursuit of AI, it is the implementation of principles for safe, ethical, legal, and interoperable AI that will reveal how coherent different national perspectives are. As of August 2021, only the United States and France have publicly issued their military AI strategies.67 Other allies, including Canada and the United Kingdom, have emerging views on responsible military AI, but little official information about how they implement their ethical risk assessments is publicly available.68 NATO’s influence in the functioning of joint operations and multinational military operations situates the Alliance to coordinate between how Allies implement ethical principles in their own national AI development. Specifically, NATO is well-situated to advocate for transparency, accountability, and data governance, which are also adoption factors that can translate into operational benefits, among other values.69 For example, these factors can promote coordination among Allies on ethical guidelines on the development and use of AI, as this will be a necessary foundation in any future joint operation that uses this technology. “The transatlantic partnership must focus on coordinating these core principles and systematic governance to ensure AI systems development aligns with the rule of law and democracy. In particular, this must ensure answering questions about human dignity, human control, and accountability … NATO remains the organization that can bring these two (U.S. and EU) together and establishes the ethical bottom line.”70 The issues of transparency and accountability will define the scope of future implementation. Many remaining questions and uncertainty will be addressed in NATO’s forthcoming AI ethical principles guidelines. But the guidelines adopted in 2021 do not address every ethical dilemma. Regarding accountability, especially, likely major questions will continue to affect the Alliance. As Assistant Secretary-General for Emerging Security Challenges David van Weel recently clarified, NATO will offer a framework of responsible use for the Allies—but the question of accountability for member states, as opposed to civilian technology manufacturers for example, is one principle that will not have an easy solution.71 International legal norms In certain respects, the legal debate mirrors much of the ethical debate surrounding AI as the two address many of the same issues. International law is a values-based system embedded in certain principles and practices agreed-upon within the international system. This section certainly identifies the complementarity of the ethical discussion surrounding AI, but it also illustrates where the legal debate can depart from the ethical considerations to address different sorts of legal challenges that face the Alliance. Lawyers, researchers, and civil society grapple with existing legal regimes relevant to military operations and the uncertainty and ambiguity surrounding automated decision-making, particularly in lethal decision-making. Thus far, the legal dialogue has been heavily anchored in the applicability of international humanitarian law (IHL), and other relevant legal regimes, to lethal autonomous weapons systems.72 IHL, also known as the laws of war or the laws of armed conflict, regulate the means and methods of warfare and, as such, is pivotal to the emergence of military technology and how existing legal structures are disrupted. The legal debate often revolves around the prospect of a “treaty ban” of LAWS.73 But the legal debate is much more nuanced than the likelihood of international treaties banning any particular weapon system. Especially because NATO is not a regulatory body, it cannot institute measures to regulate emerging technology for the Allies. Instead, NATO’s function in the legal domain may be more effective outside the traditional legal debates around emerging military technology and more embedded in fostering cooperation and coordination among military partners. Other avenues of legal regulation may fall short of an international convention or prohibition, but nevertheless factor significantly in regulating and/or delineating state policies. Additionally, non-lethal applications of AI, as well as applications of AI that do not figure into autonomous systems, also raise important legal questions under international law. Arguably, norms around non-lethal applications are more urgent as their development is more advanced, harder to define, and less controversial in integration.74 Ultimately, NATO’s facilitative power can help ensure that integration of EDTs like AI into military capabilities and into multinational coalition operations is consistent with member states’ legal obligations. One vital and unique contribution for NATO is facilitating legal interoperability among the Allies to resolve some of the most pressing legal barriers for AI implementation in future Allied operations. Legal interoperability, a subset of larger coalition interoperability, refers to the operational coordination around partner legal obligations and interpretations.75 It ensures “that within a military alliance, military operations can be conducted effectively consistent with the legal obligations of each nation.”76 Legal interoperability is a critical component of multilateral operations that has thus far been under-examined, despite its centrality to successful military operations. This is largely because “legal factors have a bearing on everything in alliances and coalition operations—from determining basic ‘troop-to-task’ considerations to decisions regarding the targets to be engaged—and the types of ordinances that may be used.”77 To enhance legal interoperability, NATO can exert its influence on how Allies can develop and deploy AI consistent with their legal obligations through its unique standardization capacities. Historically, NATO has taken significant steps to bridge the legal gap between Allies on critical procedures that bridge responsible state behavior with such “troop-to-task” considerations. One instructive example from past operations is detention policies in non-international armed conflicts.78 The promulgation of detention standards illustrates the operational significance of NATO’s common legal procedures, even for coalitions of the willing that formally operate outside NATO structures. By way of background, the U.S.-led coalition in Afghanistan had internal debates regarding the 96-hour security detention time period.79 The United States advocated extending the 96-hour rule, where coalition partners insisted adhering to the NATO standard, even though it was not a NATO operation.80 Generally the detention example illustrates NATO legal standards providing clarity to non-NATO operations; in some cases, Allies adopt NATO standards as accepted thresholds that continue to inform coalition policies beyond NATO structures and operations. Implementing AI in future military operations will almost certainly complicate legal interoperability as there is a lack of uniform standards, as in the detention example. Even some of the more basic implementation measures will garner legal uncertainty and Allies will inevitably navigate with minimal legal clarity and no standard procedures. Despite the roots of the legal debate stemming from the question of lethality, the most pressing (and urgent) legal issues will address the integration of necessary AI-enablers, such as data gathering and sharing. Furthermore, NATO has coordinated initiatives to promote awareness of Allies’ legal obligations and has a dedicated office focusing on legality. This centralizes the institutional capacity to focus on alignment not only between the policies of NATO Allies, but coherence with the international community more broadly. Among others, the NATO Legal Practitioners’ Workshop and inter-organizational dialogue between NATO, the UN, and the International Committee of the Red Cross (ICRC), the latter of which has a delegation to NATO that provides legal training and education to practitioners.81 The NATO Office of Legal Affairs (OLA) itself can also play a central role in navigating the challenges to legal interoperability. As the example of detention standards illustrates, NATO has been successful in implementing legal standards which translated into operational clarity and coalition policy outside NATO operations. As part of its focus on responsibility in its EDT agenda, NATO has opportunities to facilitate AI legal standard-setting and coalition policies to ensure safer and responsible use of AI in Allied operations. Safety and security For humans to meet ethical and legal commitments when developing and deploying AI, the systems themselves must be safe, secure, and reliable. More simply put, if humans and institutions interacting with AI do not have confidence that the systems will perform as expected, then they cannot assure that its development and deployment are responsible. This makes safety and security a key pillar of responsible AI governance for any actor.82 As this section explores for NATO in particular, safety and security are indispensable to the Alliance’s stated goals to focus its approach to EDTs in the areas of “deterrence and defense, capability development, legal and ethical norms, and arms control aspects.”83 Politically, democratic militaries using AI cannot be accountable to their citizenries nor their coalition partners if they lack mechanisms to trace and explain how their systems are reliable. Accidents and interference with AI systems could likewise create political risks for the Alliance. For example, if deepfakes and micro-targeted information attacks compromise confidence in the integrity of information used to build a common operating picture, then the operational difficulties could also erode political trust between Allies in a few key ways. In the North Atlantic Council, disagreement about the integrity of information could slow the decision-making body’s ability to react to fast-changing operational realities.84 Further, compromised AI systems may not only make it harder for forces to prevent harm to non-combatants, but also to prevent friendly fire. In this way, coalition forces arguably face even higher obligations to coordinate on the reliability of their systems, relative to adversaries and near-peer competitors that tend to operate alone. As such, responsible AI governance is not purely technical; policy alignment and strategic planning are likewise necessary to draw attention to risk management above the tactical level. Even without being attacked, governability of AI in a NATO context also means understanding how AI-enabled and autonomous systems developed by the 30 Allies—and other partners—will interact with one another. NATO has expressed interest in governability as a principle of AI “to disengage or deactivate in case of unintended behavior,”85 which echoes the U.S. Department of Defense definition of governable AI.86 Disengaging adversaries is important to maintain de-escalation measures in conflict. For NATO, interoperability between systems also relates to governable AI because allies must also consider how the interactions between the 30 Allies’ own AI-enabled and autonomous systems may result in unintended or emergent behavior.87 This means that NATO has a responsibility to coordinate activities—be they technical exchanges, standardization efforts, or training and exercises—to build confidence that the systems perform as humans intend.88 Without this coordination, the lack of interoperability of allied systems could lead to accidents, and separately, the potential loss of operational effectiveness also presents vulnerabilities for adversaries to exploit. In addition to governability, NATO and its Allies are assessing the risks that bias, attacks, and lack of interpretability can introduce in relation to the anticipated uses of a given AI system.89 In security and defense, new and heightened risks include poisoning of the information environment, deception systems and techniques, uncertainty about the performance of systems in new and unknown environments, and the possibility that tensions or accidents escalate at a faster tempo than humans and institutions can process, among others. These risks can manifest either in motivated attacks or unintentional failure modes.90 In both cases, assuring and certifying that military assets are safe and secure is important given the inherently high risk in operational environments. These operational environments include the presumption that an adversary is disrupting one’s own systems, be it by directly attacking the AI systems themselves, or disrupting the broader command, control, and communications systems under which the AI systems are operating.91 Mitigating these types of risks is typically done in testing, evaluation, validation and verification (TEVV) and in experimentation activities.92 Yet AI cannot be validated and verified the way traditional software systems are because there is no guarantee that an AI system will perform in the real world as it does in a testing environment, and because lifelong-learning systems will perform differently over their lifecycle. Having robust assurance and TEVV processes in place are also important for operators to build trust in the systems they are meant to use, as well as for citizenries and coalition partners at large to see that accountability procedures still apply. As such, building institutional procedures to govern AI safety and security is necessary to build trust in the use of the technology—as well as to develop countermeasures and defensive systems that protect against adversarial threats. NATO thus has an institutional responsibility to prevent and mitigate these intentional and unintentional failures if using AI in operations and mission support.93 As Table 69.1 shows, the Alliance also has a range of relevant entities to coordinate national approaches to AI safety and security, as well as facilitate safety measures as part of responsible use in the Alliance-wide ecosystem. NATO has an important role to play in military standardization and Allied policy planning for safe, secure, and interoperable AI. This includes the coordinating role of the Conference of National Armaments Directors and the Command, Control and Consultation Board to implement complementary acquisition processes that fuse AI adoption measures with safety responsibilities. Furthermore, entities including STO and NSO have a significant role setting the technical baseline and promulgating materiel standards that provide the technical framework for safety and security. Although their staffs are themselves small, they both convene hundreds, if not thousands, of subject matter experts in working groups. As such they both offer unique technical networks to help shape safety and security in a way that minimize risk in operations. NATO’s resources and leadership are vital to using standards and coalition policy to instill safe and secure technological development, a necessary condition to interoperable and successful future operations.

Sign in to recut, update, or save this card.