Free beta access. 50 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Stanley-Lockman & Trabucco 3/22

Zoe Stanley-Lockman & Lena Trabucco , “NATO’s Role in Responsible AI Governance in Military Affairs”, Zoe Stanley-Lockman Nanyang Technological University, Lena Trabucco University of Copenhagen, Center for Military Studies The Oxford Handbook of AI Governance Edited by Justin Bullock, Yu-Che Chen, Johannes Himmelreich, Valerie M. Hudson, Anton Korinek, Matthew Young , and Baobao Zhang Subject: Political Science, Political Institutions Online Publication Date: Mar 2022DOI: 10.1093/oxfordhb/9780197579329.013.69 – ECM

Responsible stewardship, or governance, of science and technology (S&T) requires stakeholders to change their approaches to technological development as the circumstances themselves change.22 In his book The Social Control of Technology, David Collingridge identified the double bind that makes technology governance (what he then referred to as social control) difficult: exerting social control or governing nascent technology is easy, but impossible because its evolution and eventual impacts are unknowable, and by the time the technology matures and its impact is realized, entrenched decisions will make future control more difficult.23 For now, AI remains a relatively immature technology, meaning circumstances will change as knowledge emerges and norms progressively develop. Collingridge also suggested the necessity of “corrigibility of innovation,” which refers to the “capacity to change shape or direction in response to stakeholder and public values and changing circumstances.”24 When applied to current RRI frameworks, the concept of corrigibility obligates governance stakeholders to shape the trajectory of a technology’s development and impact in ways based on social structures, both in anticipation of change and in response to decisions made in error.25 In short, stakeholders have to adopt corrigible practices to responsibly govern technology as it develops, and thus must claim their agency in guiding innovation even as technological development appears increasingly entrenched in previously made decisions and their subsequent outcomes.

This is important for AI governance because technological advancement is making AI-accelerated risks clearer, including in the military space. Risks—especially as related to AI-enabled autonomous systems, poisoning of information environments, cyberattacks, unpredictable failure modes, and emergent behavior—will evolve in form and scale as the technology matures and diffuses. If AI evolution means more entrenchment and less corrigibility, the STS foundations remind governance stakeholders how to course-correct and adapt to changing risk assessments and the overall impact of AI in the international system.26

Nevertheless, while STS scholars study how decision-making that shapes the trajectory of technological innovation becomes entrenched, the field largely rejects the premise of technological determinism. Maintaining the centrality of human agency, as exerted also through social structures and institutions, is antithetical to determinist perspectives on technology developing on its own path independent of intervention. As Allan Dafoe, another contributor to this Handbook, has argued, the STS academic community’s refusal to engage with technological determinism severely limits STS applicability to empirics.27 As discussed below, this has implications for the ability of the STS field to impart responsibility to governance stakeholders in an area such as AI and international security.

Military innovation literature

The scholars that examine the way that military stakeholders manage technology and shape its development trajectory predominantly write on military innovation.28 These scholars measure technology adoption in changes to doctrine, organizational structures, and operational concepts, rather than seeing the technology as an end in and of itself.29 From this perspective, technology subsequently shapes human and social structures and organizations. To take an example similar to Roland’s definition of technology, Jonathan Shimshoni’s concept of “military entrepreneurship” involves the active manipulation of technology, doctrine, and war plans.30 In this sense, new technology adoption has tangible and observable effects on the operational environment. Similarly, Thomas Mahnken illustrates that military services shape technology to their respective purposes, rather than the other way around.31 The purpose that this manipulation, or molding, of technology serves is the creation, and ideally sustainment, of a comparative military advantage.32

Still, the way that this military advantage is defined is relevant here because metrics of success differ from other scholarship dealing with innovation. Military innovation importantly constitutes the relationship and social structures that form between technology and military bureaucracies. Yet as a field, it does not necessarily extend these relationships to their status as stakeholders in wider technology governance regimes.33 For instance, in his review of the different schools of military innovation, Adam Grissom offers a consensus definition of military innovation that inherently links it to effectiveness in the battlespace.34 Grissom clarifies that “measures that are administrative or bureaucratic in nature, such as acquisition reform, are not considered legitimate innovation unless a clear link can be drawn to operational praxis.”35 This reinforces the idea that technology on its own does not constitute an innovation if it is not observable in military operational practice or in battlefield advantage.

The relatively narrow operational focus of military innovation scholarship means that management structures miss out on some of the uses of military power implicit in the governance of military technology. This means that both the bureaucratic entrenchment of technological advancement and the literature focusing on it do not necessarily address governance as an instrument of power in the military context. This may make sense for purely military technologies, but whether it discounts the agency that military bureaucracies have in governance of a pervasive, general-purpose technology like AI is worth separate consideration. As such, the operational measurement of the adoption and diffusion of technology as an instrument of military power likewise limits an understanding of how military technology management structures relate to governance.

Implications for military AI governance

Overall, STS offers much of the necessary groundwork for governance mechanisms and the impact of social structures on technology governance; however, it refuses to engage with technological determinism, or the independent influence of technology, that is often a driving force in military innovation. Recognizing that a comprehensive governance regime also needs to transpose to stakeholders that are engaged in the practices of governing AI, this study on NATO sees military innovation scholarship as a helpful complement to apply these STS foundations to practitioners’ perspectives. But scholarship on military innovation also has its own flaws, in that it looks at the management of technology exclusively formulated to exploit a comparative operational advantage. Measuring military innovation in relation to operational praxis makes sense to detect how military adoption of technology impacts operational excellence and upstream impacts on military strategy, but also makes it challenging for the empirics to apply to non-operational ways that military organizations exert their influence. Non-operational influence includes governance, the core topic that this chapter addresses.

Despite differences, borrowing from the layered frameworks in STS and military innovation studies still helps contextualize innovation trajectories. Indeed, select scholars have attempted to bridge the gap between the social constructivist angle in STS studies and the technologically “optimistic”36 assumptions that frame technologically deterministic undercurrents, as seen in case studies on military innovation.37 Thomas Hughes examined these undercurrents in the defense sector as part of his theory of “technological momentum,”38 which argued military organizations are subject to inaction in S&T decision-making because the entrenchment of previous investments and decisions constrain the course of future technological development. Steven Fino expands on Hughes with the idea that “technological dislocations,” are an alternative reconciliation mechanism that acknowledges that technological determinism may operate beneath the surface of a technology’s maturation trajectory, while still allowing for socially driven perturbations that “dislocate” the “otherwise logical evolutionary patterns” of that technology.39

Dafoe similarly attempts to widen the scope of technological determinism by placing it as an endpoint on a spectrum, with social constructivism on the other end. The purpose of this spectrum is to create the space for engagement with disciplines that heavily emphasize power dynamics, including military affairs and business in what he terms “military-economic adaptationism.”40 Unfortunately, both Fino and Dafoe concede that attributing agency and causality to technological developments are best “conducted after the fact”41 or “on longer timescales,” respectively.42 AI governance cannot benefit from such hindsight, as it is fundamentally a question of how to project and adapt to forces of ongoing change. For governance, this inertia places military organizations at odds with the responsiveness required to guide responsible technology governance frameworks. Our aim is not to reconcile these differences in this chapter, but rather to highlight how they frame one current governance challenge for military stakeholders such as NATO: how can they engage with the socio-technical foundations in RRI frameworks to shape, adapt to, and respond to technology-accelerated changes, while simultaneously pursuing their traditional aims of adopting technology to deter and defend?

On this note, it is worth mentioning that NATO itself has historically convened scholars from both STS and military innovation backgrounds to understand socio-technical changes to their operating environment.43 The Alliance also takes socio-technical factors into account in its S&T work on emerging technologies—including human–systems integration, technology monitoring, and forecasting work.44 This interest in socio-technical systems relating to effectiveness suggests scope for the Alliance to leverage technology governance as an instrument of its influence, as picked up in the next section.

Sign in to recut, update, or save this card.