Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

An AI ethical framework solves

Lockman and Trabucco 22 (​Zoe Stanley-Lockman is an Associate Research Fellow in the Military Transformations Programme at the Institute of Defence and Strategic Studies at the S. Rajaratnam School of International Studies in Singapore. Lena Trabucco is a dual degree candidate pursuing a PhD in political science at Northwestern University and a PhD in Law at iCourts Center of Excellence in International Courts at the University of Copenhagen. “NATO’s Role in Responsible AI Governance in Military Affairs” March 2022, ,https://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780197579329.001.0001/oxfordhb-9780197579329-e-69)ZK

Introduction The introduction of artificial intelligence (AI) as a general-purpose technology has prompted analysts and researchers to reconsider the implications for warfare. As this Handbook edition illustrates, AI has, and will continue to, shape global dialogue, policy, and governance structures in international politics, including for future military operations. In this chapter, we explore a role for the North Atlantic Treaty Organization (NATO) in the emerging military AI governance architecture. NATO (or the Alliance) is a military and political alliance among 30 contributing member states that are committed to collective security. Much of NATO’s original purpose and current core tasks arguably leave the Alliance’s role uncertain in international governance regimes contending with the impact of emerging technology on international politics.1 As global powers compete for the economic and military capabilities that AI can offer, the Alliance has the enormously challenging task of navigating varying political realities and capabilities of Allies, all while effectively recalibrating strategic relationships in the coming years. Recognizing technological change as a key variable, NATO has begun to adapt its organizational composition and strategic footing to increase the Alliance’s capacity to meet emerging security challenges for military capability development trends of both its own members and those of competitors or adversaries. New power distributions around AI and adjacent dual-use technologies are among the motivating factors causing the Alliance to reconsider whether its technological superiority may be threatened in the years ahead, as reflected in the 2019 Emerging and Disruptive Technologies (EDTs) Roadmap2 and, more recently, the NATO 2030 process.3 NATO navigates these changes and then approaches AI-accelerated changes to the international security environment in a highly political context. Notably, in 2019, French President Emmanuel Macron surprised many European counterparts by declaring NATO “brain-dead,” a warning wrapped in an even larger warning of trans-Atlantic security divisions.4 The critique that NATO is a “brain-dead” or “irrelevant” institution has existed in some form since the end of the Cold War.5 As NATO combats global perceptions of organizational irrelevance, there is a reason to push for bureaucratic adaptation to better manage technology-driven changes in the future. As such, despite some warnings to the contrary, Allies have an incentive to keep NATO a relevant military institution and ensure that it adapts to emerging threats and for future military contexts. The comment from President Macron helped prompt the NATO 2030 agenda, which is currently taking shape to increase the Alliance’s role as a political actor and as an organization with a greater focus on EDTs.6 As NATO bodies and Allies prepare for the impact of AI on future military operations, the Alliance has its own responsibility to steward AI in ways that, inter alia, promote cohesion between democratic countries, prevent risks, shore up interoperability, project deterrence, and ensure stability.7 To achieve these aims, cooperation and alignment are critical for the Alliance to maintain a competitive edge and promote further innovation in alignment with shared values. With these incentives, we argue that an examination of NATO as a governance stakeholder is due to complement other literature on how humans, social structures, and institutions impact how technology develops. More specifically, this chapter borrows from two fields of scholarship that set the theoretical foundations for how institutions such as NATO impact technological trajectories, and thus have a responsibility to govern the technology accordingly. The two fields—science, technology, and society (STS) studies and military innovation literature—have different parameters, but both explore key questions that help establish the ways in which institutions exert their influence on the development, deployment, and diffusion of technologies like AI. We argue that this influence is a form of institutional power, building on Seth Lazar’s definition of governance in this handbook. Lazar writes that governance is “the use of power to make, implement, and enforce the constitutive norms of an institution.”8 In the context of this definition, this chapter examines AI governance as an instrument of power linking NATO’s responsibility and capacity to shape the future security environment in parallel to its own organizational interests. To be sure, NATO is far from the only institution that impacts military AI governance and its security implications. Indeed, international technology governance is inherently complex because it includes diverse stakeholders in a system of “organizations, regimes, and other forms of principles, norms, regulations, and decision-making procedures” with a shared interest and responsibility in a given issue-area of world politics.9 Existing discussions of the impact of AI on international security have looked to nation-states, regional institutions like the European Union (EU), or international bodies like the United Nations Convention on Certain Conventional Weapons (UN CCW) for discussions on the military governance of AI.10 Without expanding on the role of these other stakeholders, this chapter begins to explore pressing questions for NATO and international relations scholars that illustrate NATO’s role in AI governance, which has not had a comprehensive analysis.11 To begin to fill this gap, the analysis in this chapter centers on two AI governance mechanisms that NATO has at its disposal, and subsequently explores the Alliance’s capacity to use these mechanisms to exert its influence in key pillars of AI governance. Of the many possible AI governance mechanisms for NATO, this chapter offers a deeper assessment of two: (1) strategic and policy planning and (2) standards and certification. We fashion these mechanisms as primary components that connect NATO technology governance measures and responsible AI use.12 To illustrate NATO’s capacity to govern AI, we then examine three pillars, or foundational issue areas, which we believe represent critical elements of technology governance. We argue that, within each pillar, NATO is uniquely situated to facilitate cooperation via its governance mechanisms, with a view to shaping the future of AI for the Alliance and maintaining a competitive edge. Each pillar—(1) ethics and values, (2) legal norms, and (3) security and safety—is an area where researchers and analysts have acknowledged significant governance challenges, both at a national level and for international organizations like NATO. Each pillar, discussed in depth below, illustrates NATO’s potential as a governance stakeholder that can encourage multinational alignment on policy and standards for safer and better outcomes in future operations. The rest of this chapter continues as follows. First, we establish how STS studies and scholarship on military innovation focus on different aspects of technological advancement and governance outlooks. Second, this theoretical basis is applied to NATO to provide readers with an understanding of the institution’s entities and responsibilities related to AI governance. Next, the chapter discusses ways that NATO can leverage these mechanisms to ensure responsible use of AI in military operations based on ethics, law, safety, and security. Finally, the chapter concludes with reflections on NATO’s AI governance tools and, more broadly, roles for international organizations in the AI governance space. AI Governance and Military Affairs: Tensions in Existing Literature Academic literature has long grappled with the intersection of emerging technology and security organizations.13 Two branches of literature that tackle core questions of technological trajectories and its relationship to human and social structures—a critical question of governance for military technology—are STS studies and military innovation scholarship. Although the theoretical approaches in STS and military innovation studies differ, they both share the important assumption that technology does not have its own innate logic, and instead measure technological change by its impact on social structures and interactions with humans. In other words, both fields treat technology as an enabler in broader structures. The term technology is ubiquitous enough that it does not have a single definition, but it is often defined in relation to human intention and purpose. Alex Roland describes technology as a “purposeful human manipulation of the material world” to “serve some human purpose.”14 If extending this basic idea of technology to technological innovation, then both STS studies and military-innovation scholarship lend relevant criteria. Both academic fields are also relevant because, in the policy space, AI governance stakeholders are pursuing responsible research and innovation (RRI), which comes from STS studies, and defense stakeholders are similarly focused on responsible innovation and responsible use. More traditionally, the direct study of military adoption of technology is considered in the separate scholarship of military innovation, which includes a school of thought that focus on cultural and organizational factors. Between these two fields, an interdisciplinary approach is helpful here to carry STS approaches to AI governance, including RRI, over to the space of military innovation. However, this is complicated by the reality that military organizations that see technological superiority as a core element of deterrence and defense, including NATO, engage in forms of technological determinism that STS scholars squarely reject. Respective views on technological determinism—which considers that technology shapes society as a largely autonomous process with limited human agency—thus creates a tension for governance prospects.15 To spotlight the aspects of military innovation related to governance, this section briefly expands on the overlaps and tensions between STS and military innovation literature. Science, technology, and society (STS) studies STS studies is helpful to understand how technologies such as AI develop relative to the human, social, and political structures that shape it, rather than as an independent entity to which humans have to adapt.16 In this vein, AI is not just a computational process involving software, hardware, and data,17 so much it is a socio-technical system that encompasses “human, social, and organizational factors.”18 Together, these factors enable a focus on the trajectory of technological development relative to social structures and power dynamics. STS scholars have also helped develop RRI frameworks that seek to guide technological development in anticipatory, participatory, and adaptive frameworks to achieve desirable outcomes and prevent undesirable ones.19 RRI is a structured approach to innovation in which stakeholders identify and act on their “collective commitment of care for the future through responsive stewardship of science and innovation in the present.”20 It drives civilian AI ecosystems for NATO Allies that will also indirectly affect NATO.21 Responsible stewardship, or governance, of science and technology (S&T) requires stakeholders to change their approaches to technological development as the circumstances themselves change.22 In his book The Social Control of Technology, David Collingridge identified the double bind that makes technology governance (what he then referred to as social control) difficult: exerting social control or governing nascent technology is easy, but impossible because its evolution and eventual impacts are unknowable, and by the time the technology matures and its impact is realized, entrenched decisions will make future control more difficult.23 For now, AI remains a relatively immature technology, meaning circumstances will change as knowledge emerges and norms progressively develop. Collingridge also suggested the necessity of “corrigibility of innovation,” which refers to the “capacity to change shape or direction in response to stakeholder and public values and changing circumstances.”24 When applied to current RRI frameworks, the concept of corrigibility obligates governance stakeholders to shape the trajectory of a technology’s development and impact in ways based on social structures, both in anticipation of change and in response to decisions made in error.25 In short, stakeholders have to adopt corrigible practices to responsibly govern technology as it develops, and thus must claim their agency in guiding innovation even as technological development appears increasingly entrenched in previously made decisions and their subsequent outcomes. This is important for AI governance because technological advancement is making AI-accelerated risks clearer, including in the military space. Risks—especially as related to AI-enabled autonomous systems, poisoning of information environments, cyberattacks, unpredictable failure modes, and emergent behavior—will evolve in form and scale as the technology matures and diffuses. If AI evolution means more entrenchment and less corrigibility, the STS foundations remind governance stakeholders how to course-correct and adapt to changing risk assessments and the overall impact of AI in the international system.26 Nevertheless, while STS scholars study how decision-making that shapes the trajectory of technological innovation becomes entrenched, the field largely rejects the premise of technological determinism. Maintaining the centrality of human agency, as exerted also through social structures and institutions, is antithetical to determinist perspectives on technology developing on its own path independent of intervention. As Allan Dafoe, another contributor to this Handbook, has argued, the STS academic community’s refusal to engage with technological determinism severely limits STS applicability to empirics.27 As discussed below, this has implications for the ability of the STS field to impart responsibility to governance stakeholders in an area such as AI and international security. Military innovation literature The scholars that examine the way that military stakeholders manage technology and shape its development trajectory predominantly write on military innovation.28 These scholars measure technology adoption in changes to doctrine, organizational structures, and operational concepts, rather than seeing the technology as an end in and of itself.29 From this perspective, technology subsequently shapes human and social structures and organizations. To take an example similar to Roland’s definition of technology, Jonathan Shimshoni’s concept of “military entrepreneurship” involves the active manipulation of technology, doctrine, and war plans.30 In this sense, new technology adoption has tangible and observable effects on the operational environment. Similarly, Thomas Mahnken illustrates that military services shape technology to their respective purposes, rather than the other way around.31 The purpose that this manipulation, or molding, of technology serves is the creation, and ideally sustainment, of a comparative military advantage.32 Still, the way that this military advantage is defined is relevant here because metrics of success differ from other scholarship dealing with innovation. Military innovation importantly constitutes the relationship and social structures that form between technology and military bureaucracies. Yet as a field, it does not necessarily extend these relationships to their status as stakeholders in wider technology governance regimes.33 For instance, in his review of the different schools of military innovation, Adam Grissom offers a consensus definition of military innovation that inherently links it to effectiveness in the battlespace.34 Grissom clarifies that “measures that are administrative or bureaucratic in nature, such as acquisition reform, are not considered legitimate innovation unless a clear link can be drawn to operational praxis.”35 This reinforces the idea that technology on its own does not constitute an innovation if it is not observable in military operational practice or in battlefield advantage. The relatively narrow operational focus of military innovation scholarship means that management structures miss out on some of the uses of military power implicit in the governance of military technology. This means that both the bureaucratic entrenchment of technological advancement and the literature focusing on it do not necessarily address governance as an instrument of power in the military context. This may make sense for purely military technologies, but whether it discounts the agency that military bureaucracies have in governance of a pervasive, general-purpose technology like AI is worth separate consideration. As such, the operational measurement of the adoption and diffusion of technology as an instrument of military power likewise limits an understanding of how military technology management structures relate to governance. Implications for military AI governance Overall, STS offers much of the necessary groundwork for governance mechanisms and the impact of social structures on technology governance; however, it refuses to engage with technological determinism, or the independent influence of technology, that is often a driving force in military innovation. Recognizing that a comprehensive governance regime also needs to transpose to stakeholders that are engaged in the practices of governing AI, this study on NATO sees military innovation scholarship as a helpful complement to apply these STS foundations to practitioners’ perspectives. But scholarship on military innovation also has its own flaws, in that it looks at the management of technology exclusively formulated to exploit a comparative operational advantage. Measuring military innovation in relation to operational praxis makes sense to detect how military adoption of technology impacts operational excellence and upstream impacts on military strategy, but also makes it challenging for the empirics to apply to non-operational ways that military organizations exert their influence. Non-operational influence includes governance, the core topic that this chapter addresses. Despite differences, borrowing from the layered frameworks in STS and military innovation studies still helps contextualize innovation trajectories. Indeed, select scholars have attempted to bridge the gap between the social constructivist angle in STS studies and the technologically “optimistic”36 assumptions that frame technologically deterministic undercurrents, as seen in case studies on military innovation.37 Thomas Hughes examined these undercurrents in the defense sector as part of his theory of “technological momentum,”38 which argued military organizations are subject to inaction in S&T decision-making because the entrenchment of previous investments and decisions constrain the course of future technological development. Steven Fino expands on Hughes with the idea that “technological dislocations,” are an alternative reconciliation mechanism that acknowledges that technological determinism may operate beneath the surface of a technology’s maturation trajectory, while still allowing for socially driven perturbations that “dislocate” the “otherwise logical evolutionary patterns” of that technology.39 Dafoe similarly attempts to widen the scope of technological determinism by placing it as an endpoint on a spectrum, with social constructivism on the other end. The purpose of this spectrum is to create the space for engagement with disciplines that heavily emphasize power dynamics, including military affairs and business in what he terms “military-economic adaptationism.”40 Unfortunately, both Fino and Dafoe concede that attributing agency and causality to technological developments are best “conducted after the fact”41 or “on longer timescales,” respectively.42 AI governance cannot benefit from such hindsight, as it is fundamentally a question of how to project and adapt to forces of ongoing change. For governance, this inertia places military organizations at odds with the responsiveness required to guide responsible technology governance frameworks. Our aim is not to reconcile these differences in this chapter, but rather to highlight how they frame one current governance challenge for military stakeholders such as NATO: how can they engage with the socio-technical foundations in RRI frameworks to shape, adapt to, and respond to technology-accelerated changes, while simultaneously pursuing their traditional aims of adopting technology to deter and defend? On this note, it is worth mentioning that NATO itself has historically convened scholars from both STS and military innovation backgrounds to understand socio-technical changes to their operating environment.43 The Alliance also takes socio-technical factors into account in its S&T work on emerging technologies—including human–systems integration, technology monitoring, and forecasting work.44 This interest in socio-technical systems relating to effectiveness suggests scope for the Alliance to leverage technology governance as an instrument of its influence, as picked up in the next section. NATO’s Mechanisms to Govern AI NATO’s increasing interest in EDTs introduces the need to consider how governance priorities can help reinforce the Alliance’s influence. The STS and military innovation literature provide the theoretical foundations for NATO’s stewardship of AI as they place attention on “the role that institutions play in shaping technological trajectories.”45 As AI development continues, the actions that NATO and its members take will have important implications for their capacity to adopt, respond to, and shape their future operating environment. Particularly for democracies, this confers to military stakeholders a dual responsibility to prevent and manage risks, as well as to proactively shape their approach to technological development anchored in democratic values and security. As a multinational alliance with an incentive to drive cooperation and alignment, NATO is situated to define and operationalize norms, as well as promote standards that help shape the contours of future military effectiveness and technological competition. In a RRI framework, not only is this an institutional role, but it also becomes an institutional responsibility. To apply this responsibility to NATO’s stewardship of AI, the institutional interplay between technology, structure, and concepts is a form of socio-technical system with important implications for AI governance because they link the ways that an institution uses its power to adopt and shape AI trajectory to its respective ends. Already, several mechanisms are built into military bureaucracies to ensure that technology is adopted in alignment with responsible engineering practices and responsible state behavior.46 The Alliance is organized to harmonize between Allies so that their contributions enhance military effectiveness and political cohesion between like-minded democracies. We argue that these effectiveness-centric mechanisms likewise empower NATO to exert its influence in technology governance. More specifically, this entails the Alliance helping steward technological development for a more predictable strategic environment and enhanced democratic clout around the exploitation of technology reinforcing rule of law. For NATO, we focus on strategic and policy planning, as well as standards and certification because they reflect the Alliance’s particular strengths and interests in S&T. These practices are relevant to governance insofar as they exemplify an institution’s power to shape the trajectory of technological development—but this selection is by no means exhaustive.47 Instead, our aim is to explore how these mechanisms are operationalized at the Alliance level. In this vein, Table 69.1 dissects the role that its various bodies play in managing technology, promulgating and operationalizing standards, and leading change through policy. The role of NATO in this equation is largely shaped by its members’ own approaches to technology, and member-state-driven processes are complemented by “policy entrepreneurs” and technical experts in the International Staff and related bodies.48 Table 69.1 does not list the ways that AI affects the various functions of NATO, but rather spotlights the entities that together operationalize AI governance through cumulative processes on policy and standardization. Strategic and policy planning NATO structures around strategic and policy planning both set Allied ambitions and priorities and have the competency to implement them through its many consultative bodies, coordination formats, and albeit to a lesser extent, technology foresight capacities. NATO has facilitative power among Allies, both for defense planning and for the conduct of operations. A cornerstone in modern architecture of international security is coalition warfare—or, more broadly, joint operations. Working with military partners has become a critical feature of modern security policy, where there is more power in enhancing numbers, but also in having allies that lend political and practical legitimacy to deterrence and operations.49 NATO is vital to that effort for many reasons, but also because NATO’s facilitative power is significant to promote coordination and cooperation. Simply put, partners and allies are a necessary feature of modern military behavior, and strategic and policy planning are necessary functions to encourage and underpin cohesion in alliance settings. This is important for AI governance because the nature of AI poses new strategic challenges and will require multilateral approaches and some degree of cohesion to effectively incorporate RRI frameworks in policy planning. As such, the necessity of working with security partners extends to the AI-policy frontier. A number of NATO entities carry out strategic and policy planning, recognizing the importance of policy alignment to sustain political strength and military effectiveness. As relates to S&T, allies’ representations to NATO, defense ministries, and policy entrepreneurs from the relevant entities summarized in Table 69.1 support and negotiate how the Alliance approaches EDTs. NATO’s strategic documentation and forward-looking policy analysis incorporates hints of technological determinism, including noting how technological change inevitably shapes the future strategic and operating environment. Further, the connections between technology and competitive advantage over adversaries and competitors are embodied in the Alliance’s desire to maintain its “technological edge” as the “foundation upon which NATO’s ability to deter and defend against potential threats ultimately rests.”50 This places technology squarely within NATO’s core purpose of deterrence and defense—and while this signals NATO’s express commitment to technology through these channels, this reliance on technology also obscures whether NATO’s governance capacity will be adaptive, anticipatory, or participatory. This position of technological determinism may result in more limitations for AI governance. Standards and certification To maintain its relevance in a security architecture increasingly concerned with the way that technology shifts power dynamics and scales threats to international security, NATO has an incentive to foster cooperation, promote standards of practice, and incentivize Allied AI harmonization. It is strategically salient to facilitate a dialogue and engagement among Allies on AI, but it is practically important to use NATO’s position to facilitate Allied cooperation regarding standards to project the Alliance’s ability to interoperate in future operations. NATO standards aim to enhance interoperability among partners and successful implementation of strategy. More specifically, standards and certification are used to establish and implement requirements aligned with safe development and responsible use of technology. In addition to purely technical standards, NATO has operational standards that specify “conceptual, organizational or methodological requirements to enable materiel, installations, organizations or forces to fulfil their functions or missions.”51 In line with the definitions from STS and military innovation scholarship, standards can thus be seen as a mechanism to translate responsibility-derived state and organizational AI policy into actionable functions. In fact, NATO has set certain standards for the Allies and these standards subsequently become the norm. Within NATO, it is the NATO Standardization Office (NSO) that coordinates thousands of experts to align technological development with military requirements that can help enhance effectiveness, interoperability, and cohesion.52 While the NSO is primarily responsible for setting standards, other NATO entities—including in the NATO Science and Technology Organization (STO)—play important roles in implementing them and coordinating between national approaches.53 Certification frameworks and the promulgation of best practices can similarly help incentivize the transposition of RRI into military organizations, even if standardization is by no means a purely military governance tool. Both mechanisms, strategic policy planning and standards and certification, provide options for NATO to participate in AI governance regimes focusing on international security. NATO’s operationalization of these tools may hold important implications while implementing successful AI governmental regimes for Allies and other defense stakeholders. In the next section, we consider each mechanism within foundational issues, or pillars, to illustrate NATO’s role in AI governance. NATO and Technological Change: Three Pillars of AI Governance This section considers three pillars where NATO has procedures and competency to operationalize AI governance through both mechanisms of policy alignment and standards, and enhance security in the international environment. The pillars reflect foundational issue areas constitutive of governance but are also issue areas where previous scholars have cautioned as particularly challenging in the AI governance space. The three pillars—(1) ethics and values, (2) legal norms, and (3) safety and security—are meant to illustrate three conditions for NATO to facilitate policy and standards harmonization. Importantly, these pillars are not exhaustive areas in which NATO will need to consider governance structures to responsibly implement AI technology, but rather highlight particular issues that researchers and analysts acknowledge as significant hurdles in navigating AI governance (see Table 69.2).54 The first pillar considers NATO’s role in the evolution of ethical and values-driven AI. One ongoing debate regarding AI as a military technology is the ethical implications and baseline values the Allies, and others, want infused in the development and adoption of AI. The Allies themselves lack uniform consensus on numerous, substantial ethical questions on the use of AI, as most clearly seen in the adjacent area of the ethics of autonomy in weapons systems including lethal autonomous weapon systems (LAWS). In this discussion, we spotlight NATO’s role in facilitating and shaping ethical harmonization as an operational requirement to ensure successful future missions. The second pillar examines legal norms as a domain wherein legal uncertainty regarding AI has tangible implications for Allied legal interoperability, a subset of larger coalition interoperability. Thus far, the legal debate regarding AI has been largely fixed on the issue of a treaty banning the use of LAWS. In this section, we advocate for a more nuanced legal picture in which NATO can facilitate legal coordination and tackle some of the foundational legal issues which will prevent successful legal interoperability in future operations. The third pillar identifies safety and security of AI systems as prerequisite to trustworthy and responsible AI in any context, but especially so for the conduct of military activity. At the NATO level, Allied forces must ensure their systems interoperate safely and predictably both to ensure effective command and control (C2) internally, and to prevent disruptions from attacks. It is a foundational facet of coordination that shows the overlap between NATO interests in military effectiveness and incentivization for responsible innovation.

Sign in to recut, update, or save this card.