Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Open evidence library

Search the archive.

Current and older evidence is free during beta. Sign in for unlimited card access and full source downloads.

Matching cards
2022 MNDI Affirmative
TEVV AFF – MNDI HPA › Contention One Extensions › AT JAIC Solves Now

Enhancing the JAIC is necessary for solvency – they coordinate across different services and maintain focus on the message and mission for AI

Tarraf, Shelton, Parker 2019 - Senior Information Scientist, Senior Engineer, and physical Scientist at the RAND Corporation [ Danielle C., William, Edward, et al RAND Cooperation, “The Department of Defense Posture for Artificial Intelligence”, , Acc 6/18/22, M.A.]

Organization We begin by revisiting DoD’s vision for AI and the means of achieving that vision. As we noted in Chapter Four, there is a lack of clarity about the raison d’être of the JAIC, and how the specific mandate and roles it has been assigned—and the visibility and authorities it has been given—support that. DoD needs to be consistent in its intent, actions, and messaging. Our first strategic recommendation therefore addresses DoD’s vision for AI and the governance structures that would support this vision, as articulated in the DoD AI strategy. Recommendation S-1: DoD should adapt AI governance structures that align authorities and resources with the mission of scaling AI. 66 The Department of Defense Posture for Artificial Intelligence As we noted in Chapter Four, DoD’s vision for AI and the scale, urgency, and unity of efforts conveyed in that vision are at odds with the visibility, authorities, and resources it has provided to the JAIC— the focal point of DoD AI. DoD needs to develop governance and organizational structures that align authorities and resources with its vision of scaling AI across DoD. The insights gathered from our industry interviews and the supporting change management literature (see section “Industry: Organization” in Appendix C) lead us to believe there is value in, if not strict necessity of, a centralized effort supported at the highest levels with long-term funding commitments to institute organizational change and scale AI across DoD. Indeed, one of our industry interviewees even noted that centralization at onset was key to their organization’s success, and premature decentralization of effort would have likely been detrimental (see section “Industry: Organization” in Appendix C).1 Starting from that premise, we highlight two possible options for organizational and governance structures, and the rationale for them, although noting that other options might be viable as well, subject to further study. The first option would likely require congressional support to execute, while the second can be executed without, as it aligns with current DoD procedures and organizational structures. Option 1: Enhance the visibility and authorities of the JAIC to enable it to carry out its mission of scaling AI and its impact across DoD, including budgetary and workforce authorities over the military services. Option 2: Take a two-pronged organizational approach as follows: • Establish a JAIC council chaired by the JAIC director and consisting of one AI leadership representative from each service.2 1 We note that this comment, and those of other industry interviewees, imply that if the effort to scale AI across an organization is successful, its natural ending point might be the sunset of the centralized entity that drove the transformation as AI capabilities are diffused across the organization. We therefore expect the JAIC’s role to evolve, though we expect that to happen along a longer timeline (ten or more years), based on our assessment of the state of AI technologies in Chapter Three. 2 Regardless of which governance structure is instituted, if any, establishment of a JAIC council, as described, could facilitate coordination between the JAIC and the services. Recommendations 67 • Establish or reinforce a centralized AI coordination and investment organization within each of the services, with appropriate visibility and authorities, to facilitate scaling AI and its impact across the service, and to promote mandated coordination with the JAIC. In either option: • The DSD should provide the JAIC director with opportunities, at least annually, to present and be heard at the Deputy’s Management Action Group (DMAG) forum (or whichever Deputy Secretary–level forum performs the functions of the DMAG). 3 The rationale for Option 1 is as follows: There is evidence to support that DoD has taken the right approach in establishing the JAIC as a centralized focal point for DoD’s AI strategy (see Chapter 4, Organization). The evidence also suggests that the JAIC, pending initial success, will need to continue in that role for several years because of the expected timeline for AI deployment across enterprise, missionsupport, and operational AI (Chapter Three).

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › Contention One Extensions › AT JAIC Solves Now

JAIC does not solve now – it lacks authority for budgeting.

Tarraf, Shelton, Parker 2019 - Senior Information Scientist, Senior Engineer, and physical Scientist at the RAND Corporation [ Danielle C., William, Edward, et al RAND Cooperation, “The Department of Defense Posture for Artificial Intelligence”, , Acc 6/18/22, M.A.]

The JAIC lacks the authorities to carry out its present role. At its core, the JAIC’s overarching mission can be distilled to this: Scale AI and its impact across DoD. This mission and its present scope—as defined by the summary of the DoD AI strategy and the memo establishing the JAIC—are extensive, while the JAIC’s current authorities are limited. In particular, the JAIC is expected to synchronize DoD AI activities and coordinate AI initiatives totaling more than $15 million annually. It is unclear whether the JAIC has any mechanisms for enforcing these directives, because it does not have the authorities to direct investments or to halt programs or activities that are deemed to be misaligned with DoD’s strategy (a fact we learned through multiple DoD interviews). In short, the JAIC does not have directive or budget authorities, and that critically limits its ability to synchronize and coordinate DoD-wide AI activities to enact change. Currently, it can catalogue these activities, but it is unclear how doing so would help scale AI across DoD. Of course, that assumes that what constitutes an AI activity is known. However, it is not currently clear how the determination of what constitutes an AI initiative or activity is made, by whom, and whether that determination is consistent across DoD.8

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › Contention One Extensions › AT NATO Guidelines Solve Now

NATO’s guidelines are just principles – they do not implement policies that back them up – that is the plan.

Stanley-Lockman and Christie, 2021 - Innovation Officer in the Emerging Security Challenges Division in NATO and former Deputy Head of Innovation in NATO’s International Staff [Zoe and Edward, NATO Review, October “An Artificial Intelligence Strategy for NATO” , BK]

Adopting AI in the defense and security context also calls for effective and responsible governance, in line with the common values and international commitments of Allied nations. To that end, Allied governments have committed to Principles of Responsible Use as a key component of NATO’s AI Strategy. Allies and NATO commit to ensuring that the AI applications they develop and consider for deployment will be in accordance with the following six principles: Lawfulness: AI applications will be developed and used in accordance with national and international law, including international humanitarian law and human rights law, as applicable. Responsibility and Accountability: AI applications will be developed and used with appropriate levels of judgment and care; clear human responsibility shall apply in order to ensure accountability. Explainability and Traceability: AI applications will be appropriately understandable and transparent, including through the use of review methodologies, sources, and procedures. This includes verification, assessment and validation mechanisms at either a NATO and/or national level. Reliability: AI applications will have explicit, well-defined use cases. The safety, security, and robustness of such capabilities will be subject to testing and assurance within those use cases across their entire life cycle, including through established NATO and/or national certification procedures. Governability: AI applications will be developed and used according to their intended functions and will allow for: appropriate human-machine interaction; the ability to detect and avoid unintended consequences; and the ability to take steps, such as disengagement or deactivation of systems, when such systems demonstrate unintended behaviour. Bias Mitigation: Proactive steps will be taken to minimise any unintended bias in the development and use of AI applications and in data sets. Having agreed to adopt these mutually reinforcing principles, the task now turns to translating them into principled action. As such, NATO’s role in operationalising these principles will involve efforts that similarly tackle different aspects of the technology’s lifecycle. Building the principles of responsible use into the front end of AI development is important because, the later they are considered, the harder it may be to ensure they are upheld. Ensuring a full life-cycle approach also depends on multi-stakeholder engagement because responsibility is diffused amongst the policymakers, designers, developers, and testers, as well as operational end users that engage in AI development and use. For NATO, this is relevant because various entities play an active role in AI integration, and because the Alliance can encourage coherence with national AI developments.

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › Contention One Extensions › AT DOD AI Strategy Solves Now

The US has not committed to AI safety – there were promises but no concrete action – this creates the risk of accidental war.

Lewis 2019 - project lead for the DOD’s Joint Lessons Learned studies [Larry, “AI Safety: Charting out the High Road”, War on the Rocks Dec 19, https://warontherocks.com/2019/12/ai-safety-charting-out-the-high-road/ LMSi]

Concerns about how governments can leverage AI also extend to the waging of war. Two major concerns about the application of AI to warfare are ethics (is it the right thing to do?) and safety (will civilians or friendly forces be harmed, or will the use of AI lead to accidental escalation leading to conflict?). With the United States, Russia, and China all signaling that AI is a transformative technology central to their national security strategy, with their militaries planning to move ahead with military applications of AI quickly, should this development raise the same kinds of concerns as China’s use of AI against its own population? In an era where Russia targets hospitals in Syria with airstrikes in blatant violation of international law, and indeed of basic humanity, could AI be used unethically to conduct war crimes more efficiently? Will AI in war endanger innocent civilians as well as protected entities such as hospitals? BECOME A MEMBER To be clear, any technology can be misused in war. A soldier could use a rock to commit a war crime. A simple, low-tech land or sea mine can be used indiscriminately and endanger civilians if it is used in the wrong way. A transformative technology like AI can be used responsibly and safely, or it could fuel a much faster race to the bottom. The United States has declared it will take the high road with military applications of AI. For example, the Department of Defense AI strategy has “AI ethics and safety” as one of the its fundamental lines of effort. And this is not an empty promise: The Defense Innovation Board just released its principles for ethical military use of AI, marking a year-long, deliberate initiative drawing in AI experts, ethicists, and the general public. By this laudable effort, the United States has shown leadership in the responsible and principled use of this technology in war. But there is something missing: The AI strategy commitment was to ethics and safety. To date, the Department of Defense has not shown a similar, concerted focus on AI safety. Despite commitments made to the international community and in its own AI strategy, the Pentagon has done little to act on promises to address safety risks unique to the technology of AI or to use AI to enhance safety in conflict. My recent research has shown that this inaction creates risks to those on the battlefield, for civilians and combatants alike, and increases the likelihood of accidental escalation and conflict. In an era where the technology of AI can so easily be exploited by governments to violate the principles of humanity, the United States can demonstrate the high road is possible, but to do so it needs to keep its promises: to address safety risks intrinsic to AI and to search for ways to use AI for good.

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › Contention One Extensions › Extend No Arms Race

Framing AI as an arms race assumes that AI is only a weapon – it is beneficial in many other contexts

Scharre 2021 – Director of the Technology and National Security Program at the Center for a New American Security [Paul, 6/28/21,https://tnsr.org/2021/06/debunking-the-ai-arms-race-theory/, “Debunking the AI Arms Race Theory”, 6/18/22, LND]

As Heather Roff has written, the arms race framing “misrepresents the competition going on among countries.” To begin with, AI is not a weapon. AI is a general-purpose enabling technology with myriad applications. It is not like a missile or a tank. It is more like electricity, the internal combustion engine, or computer networks. General-purpose technologies like AI have applications across a range of industries. Wired magazine co-founder Kevin Kelly has argued that it “will enliven inert objects, much as electricity did more than a century ago. Everything that we formerly electrified we will now cognitize.” Nations may very well be in a technology race to adopt AI across a range of industries. AI will help to improve economic productivity and, by extension, economic and military power. During the industrial revolution, early adopters of industrial technology significantly increased their national power. From 1830 to 1890, Britain and Germany, which were both early industrializers, more than doubled their per capita gross national product while Russia, which lagged in industrialization, increased its per capita gross national product by a mere 7 percent over that 60-year period. These technological advantages led to increased economic and military power, most notably for Europe relative to the rest of the world. In 1790, Europe (collectively), China, and India (including what is now Pakistan and Bangladesh) held roughly the same shares of global manufacturing output, with Europe and India each holding about one-quarter of global manufacturing output and China holding roughly one-third. They all had approximately equivalent levels of per capita industrialization at that time. But the industrial revolution skyrocketed European economic productivity. By 1900, Europe collectively controlled 62 percent of global manufacturing output, while China held only six percent and India less than two percent. These economic advantages translated into military power. By 1914, Europeans occupied or controlled over 80 percent of the world’s land surface. Being ahead of the curve in adopting AI is likely to lead to significant national advantages. Although AI can increase military capabilities, the more consequential advantages over the long term may come from non-military AI applications across society. Long-term benefits from AI could include increased productivity, improved healthcare outcomes, economic growth, and other indicators of national well-being. Increasing productivity is especially significant because it has a compounding effect on economic growth. Over the long term, technological progress is the main driver of economic growth.

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › Contention One Extensions › Extend No Arms Race

AI is different from other Arms Races – it is not exclusively military, which means there is less advantage to moving first

Horowitz, 2018 – Professor of Political Science at UPenn [Michael, September “The Algorithms of August: The AI arms race won't be like previous competitions, and both the United States and China could be left in the dust,” https://foreignpolicy.com/2018/09/12/will-the-united-states-lose-the-artificial-intelligence-arms-race/ 6/18/22 MD]

AN  ARMS RACE IS COMING. It is Unlikely to play out in the way that the mainstream media suggest, however: as a faceoff between the United States and China. That's because AI differs from the technologies, such as nuclear and battleships, that have been the subject of arms races in the past. After all, AI is software--not hardware. Because AI is a general purpose --more like the combustion engine or electricity than a weapon--the competition to develop it will be broad, and the line between its civilian and military uses will be blurry. There will not be one exclusively military AI arms race. There will instead be many AI arms races, as countries (and, sometimes, violent nonstate actors) develop new algorithms or apply private sector algorithms to help them accomplish particular tasks. In North America, the private sector invested some $15 billion to $23 billion in AI in 2016, according to a McKinsey Global Institute report. That's more than 10 times what the U.S. government spent on unclassified AI programs that same year. The largest share came from companies such as Google and Microsoft, as well as a number of smaller private firms, not from government-funded defense research. This reverses the dynamic from the Cold , when government investments led to private sector innovation and produced technologies such as GPS and the internet. China says it already holds more than 20 percent of patents in the field and plans to build its AI sector to be worth $150 billion by 2030. But while Beijing and Washington are the current leaders in this race, they are not the only competitors. Countries around the world with advanced technology sectors, from Canada to France to Singapore, also have the potential to make great strides in AI (or build on lower-level advances made by others). While this diffusion means that many more countries will have a stake in the regulation of AI, it also means that many more governments will have incentives to go it on their own. UNLIKE THE DEVELOPMENT of a Stealth bomber, which has only military applications, basic AI research has both military and civilian uses, which makes it much harder to keep research secret and thereby sustain a large first-mover advantage. The dual-use character of many developments in AI creates an incentive to promote their release and spread to the general public. That means companies can co-opt advances made by market leaders--especially lower-level advances that do not require significant computing hardware.

Autonomous weapons won’t cause an Arms Race – just because countries develop them doesn’t mean that they are in a hostile race.

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › Contention One Extensions › Extend No Arms Race

AI is not an Arms Race – spending is not enough to warrant the Arms Race title – it is just military modernization

Scharre 2021 – Director of the Technology and National Security Program at the Center for a New American Security [Paul, 6/28/21,https://tnsr.org/2021/06/debunking-the-ai-arms-race-theory/, “Debunking the AI Arms Race Theory”, 6/18/22, LND]

The scale of military AI spending, at least at present, is nowhere near large enough to warrant the title of “arms race.” Of course, AI can also be used for weapons. Militaries around the world are actively working to adopt AI to improve their military capabilities. Yet the militarization of AI does not, at present, meet the traditional definition of an arms race, despite the rhetorical urgency of many national leaders. Michael D. Wallace, in his 1979 article “Arms Races and Escalation,” defined an arms race as “involving simultaneous abnormal rates of growth in the military outlays of two or more nations” resulting from “the competitive pressure of the military itself, and not from domestic forces exogenous to this rivalry.” Wallace further stated that the concept of an arms race only applied “between nations whose foreign and defense policies are heavily interdependent” and who have “roughly comparable” capabilities. AI is being adopted by many countries around the globe. Arguably at least some of the dyads, such as the United States and China, meet Wallace’s definition in terms of being nations with “roughly comparable” capabilities, locked in competition, “whose foreign and defense policies are heavily interdependent.” However, AI fails the arms race test in the critical area of spending. Wallace distinguished arms races from the normal behavior of states to improve their military forces. A state that adopts a new technology and modernizes its military forces is not automatically in an arms race, under Wallace’s definition, even if the modernization is aimed at competition with another country. The decisive factor in qualifying as an arms race, according to Wallace, is the rate of growth in defense spending. Wallace characterized arms races as resulting in abnormally large growth rates in defense spending, beyond the historical average of 4 to 5 percent annual growth (in real dollars). In an arms race, annual growth rates are above 10 percent or even as high as 20 to 25 percent. Other scholars define arms races using different quantitative thresholds — and some definitions lack clear quantitative thresholds at all — but the existence of rapid increases in defense spending or military forces above normal levels is a common criterion in the scholarly literature on arms races. Arms races result in situations in which two or more countries are locked in spiraling defense spending, grabbing ever-greater shares of national treasure often with little to no net gain in relative advantage over the other. Classic historical examples include the Anglo-German naval arms race prior to World War I and the U.S.-Soviet nuclear arms race during the Cold War. Military AI spending today clearly does not meet these criteria of abnormally large growth rates in defense spending. AI defense spending is difficult to calculate due to the general-purpose nature of AI technology. Unlike ships or ballistic missiles, AI systems cannot be easily counted. Nevertheless, even crude estimates of defense spending show that military AI investments are nowhere near large enough to constitute an arms race. An independent estimate by Bloomberg Government of U.S. defense spending on AI identified $5 billion in AI-related research and development in fiscal year 2020, or roughly 0.7 percent of the Department of Defense’s over $700 billion budget. The scale of military AI spending, at least at present, is nowhere near large enough to warrant the title of “arms race.” (Adding in private sector spending, which constitutes the bulk of AI investment, would lead to larger figures but would further belie the claim of an “arms” race since most private sector AI investment is not in weapons.)

There is no global sprint or arms race – “Arms Race” claims overexaggerate normal military behavior and ignore actual spending.

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › Contention One Extensions › Extend No Arms Race

Claims of an AI arms race are overblown – the AI adoption process is routine continuation of modernization

Horowitz and Scharre, 2021 - Director of the Emerging Capabilities Policy Office in the Office of the Under Secretary of Defense for Policy and  Vice President and Director of Studies at CNAS [Michael, Paul, January 12 2021, “AI and International Stability: Risks and Confidence- Building Measures”, , Acc 6/18/22. M.A.]

AI is a general-purpose technology akin to computers or the internal combustion engine, not a discrete technology like missiles or aircraft. Thus, while concerns of an “AI arms race” are overblown, real risks exist. Additionally, despite the rhetoric of many national leaders, military spending on AI is relatively modest to date. Rather than a fervent arms race, militaries’ pursuit of AI looks more like routine adoption of new technologies and a continuation of the multi-decade trend of adoption of computers, networking, and other information technologies. Nevertheless, the incorporation of AI into national security applications and warfare poses genuine risks. Recognizing the risks is not enough, however. Addressing them requires laying out suggestions for practical steps states can take to minimize risks stemming from military AI competition. One approach states could take is adopting confidence-building measures (CBMs): unilateral, bilateral, and/or multilateral actions that states can take to build trust and prevent inadvertent military conflict. CBMs generally involve using transparency, notification, and monitoring to attempt to mitigate the risk of conflict. There are challenges involved in CBM adoption due to differences in the character of international competition today versus during the Cold War, when CBMs became prominent as a concept. However, considering possibilities for CBMs and exploring ways to shape the dialogue about AI could make the adoption of stability-promoting CBMs more likely.

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › Contention One Extensions › Extend Inherency – TEVV

Current TEVV fails – new techniques needed to address complexities in AI tech

Scharre, 2016 -- Vice President and Director of Studies at CNAS [Paul, Feb 2016, Center for New American Security, “Autonomous weapons and operational risk”, https://s3.us-east-1.amazonaws.com/files.cnas.org/documents/CNAS_Autonomous-weapons-operational-risk.pdf?mtime=20160906080515&focal=none, Acc 6/21/22, M. A.]

System failures: System failures occur not from the breakdown of any one given part, but from unanticipated interactions between elements of a system. Verifying all possible combinations of the internal workings of the system becomes increasingly difficult as the system’s complexity increases. o A recent report on autonomy by the U.S. Air Force Office of the Chief Scientist highlighted the need for new techniques for the verification and validation of autonomous software as a “critical” issue for the Air Force. “Traditional methods … fail to address the complexities associated with autonomy software … There are simply too many possible states and combination of states to be able to exhaustively test each one.”14 11 Emergent behavior could come from individual systems or from groups or swarms of simpler systems coordinating their actions together, similar to ants, termites, or bees. For more on military applications of swarming, see Paul Scharre, “Robotics on the Battlefield – Part II: The Coming Swarm,” Center for a New American Security, October 2014,

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › Contention One Extensions › Extend Inherency – TEVV

Current TEVV is not enough – assurance is an afterthought for most AI programs

Batarsheh et al. 2021 - Professor of Electrical and Computer Engineering, Virginia Polytechnic Institute [Feras, Laura Freeman Director, Intelligent Systems Division – National Security Institute Research Associate Professor, Chih Hao Huang, Data Analyst at Turing Research. April 26 “A survey on artificial intelligence assurance” https://link-springer-com.proxy.lib.umich.edu/article/10.1186/s40537-021-00445-7#Sec8]

Applications of AI such as facial recognition using deep learning have become commonplace. Deep learning models are often exposed to adversarial inputs (such as deep-fakes), thus limiting their adoption and increasing their threat [145]. Unlike conventional software, aspects such as explainability (unveiling the blackbox of AI models) dictate how assurance is performed and what is needed to accomplish it. Unfortunately however, similar to the software engineering community’s experience with testing, ensuring a valid and verified system is often an afterthought. Some of the classical engineering approaches would prove useful to the AI assurance community, for instance, performing testing in an incremental manner, involving users, and allocating time and budget specifically to testing, are some main lessons that ought to be considered. A worthy recent trend that might aid majorly in assurance is using AI for testing AI (i.e., deploying intelligence methods for the testing and assurance of AI methods). Additionally, from a user’s perspective, recent growing questions in research that are relevant to assurance pose the following concerns: how is learning performed inside the blackbox? How is the algorithm creating its outcomes? Which dependent variables are the most influential? Is the AI algorithm dependable, safe, secure, and ethical? Besides all the previously mentioned assurance aspects, we deem the following foundational concepts as highly connected, worthy of considering by developers and AI engineers, and essential to all forms of AI assurance: (1) Context: refers to the scope of the system, which could be associated with a timeframe, a geographical area, specific set of users, and any other system environmental specifications (2) Correlation: the amount of relevance between the variables, this is usually part of exploratory analysis, however, it is key to understand which dependent variables are correlated and which ones are not, (3) Causation: the study of cause and effect; i.e., which variables directly cause the outcome to change (increase or decrease) in any fashion, (4) Distribution: whether a normal distribution is assumed or not. Data distribution of the inputted dependent variables can dictate which models are best suited for the problem at hand, and (5) Attribution: aims at allocating the variables in the dataset that have the strongest influence on the outcomes of the AI algorithm.

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › Contention One Extensions › Extend Inherency – DOD

The DOD AI Strategy fails due to a lack of a value metric to assess progress

Tarraf, Shelton, and Parker 2019 - Senior Information Scientist, Senior Engineer, and physical Scientist at the RAND Corporation [ Danielle C., William, Edward, et al RAND Cooperation, “The Department of Defense Posture for Artificial Intelligence”, , Acc 6/18/22, M.A.]

DoD lacks baselines and metrics in conjunction with its AI vision. Baselines and metrics are important for two reasons. First, they are a means of assessing and enhancing progress toward DoD’s vision and of managing expectations (see section “Adoption and Scaling of Unmanned Aircraft Systems” in Appendix D and section “AI History in DoD” in Appendix D). Second, metrics are needed to demonstrate value and secure continued leadership support as progress is made in institutional transformation (see section “Industry: Organization” in Appendix C); this is particularly important in ensuring continued support at the highest levels of decisionmaking, both within and outside DoD. The summary of the 2018 DoD AI strategy lays out a vision for institutional transformation through AI, the assignment of an organization to be a focal point for that transformation, and a set of activities associated with the vision. However, the strategy does not articulate baselines, metrics, or quantifiable measures of value or success.

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › Contention One Extensions › Extend Inherency – DOD

DOD has not committed to improve TEVV – this prevents the US from adopting safe AI

Flournoy, Haines et al 2020 - former Under Secretary of Defense for Policy and Director of National Intelligence [Michèle and Avril, Gabrielle Cheftiz Special Assistant to the Under Secretary of Defense for Policy October, “Building Trust through Testing Adapting DOD’s Test & Evaluation, Validation & Verification (TEVV) Enterprise for Machine Learning Systems, including Deep Learning Systems” https://cset. georgetown.edu/wp-content/uploads/Building-Trust-Through-Testing.pdf Acc 6/23/22 JZ]

The Defense Department needs to reform its existing testing and verification system—its methods, processes, infrastructure, and workforce—in order to help decision-makers and operators understand and manage the risks of developing, producing, operating, and sustaining AI-enabled systems. Several DOD reports and policy documents identify TEVV as a barrier to AI adoption and call for increased research into new methodologies, including the Pentagon’s AI Ethics Principles1 and AI Strategy,2 which states, “we will invest in the research and development of AI systems that are resilient, robust, reliable, and secure; we will continue to fund research into techniques that produce more explainable AI; and we will pioneer approaches for AI test, evaluation, verification, and validation.” However, DOD has yet to translate this stated goal into a real plan of action. Advancing the Defense Department’s TEVV enterprise for ML/DL systems is critical for several reasons. First, developing an effective TEVV approach that is sufficiently predictive of performance is critical to building the trust in these systems necessary to deploy and leverage these capabilities at scale. The United States has already seen this dynamic with nuclear power, for example, where lost trust in the technology has prevented policymakers from harnessing nuclear power for clean energy

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › 1AC › 1AC - Contention Two

Reforming the DOD TEVV system will allow coordination and continuous lifecycle testing – these are essential for AI

Flournoy, Haines et al 2020 - former Under Secretary of Defense for Policy and Director of National Intelligence [Michèle and Avril, Gabrielle Cheftiz Special Assistant to the Under Secretary of Defense for Policy October, “Building Trust through Testing Adapting DOD’s Test & Evaluation, Validation & Verification (TEVV) Enterprise for Machine Learning Systems, including Deep Learning Systems” https://cset. georgetown.edu/wp-content/uploads/Building-Trust-Through-Testing.pdf Acc 6/23/22 JZ]

For ML/DL, the Defense Department will need to replace its classic approach to TEVV of formulating a T&E Master Plan for a given capability up front with a more automated, iterative, and continuous approach to testing in line with DevSecOps. Assuring that ML/DL systems function as expected and do not engage in behaviors outside their intended use and operational parameters will require testing across the system’s entire life cycle—from development to operational deployment to sustainment. It will also require new methods for capturing lessons learned and integrating these into iterative development and testing. Because of the difficulty predicting and binding system performance, one should consider every deployment of an ML/DL system as an experiment and opportunity to collect data and insight on performance.16 To support this approach, DOD will need to expand coordination between program managers and testers to ensure testing milestones are built in throughout the acquisition program. Program managers often see TEVV as an obstacle to be surmounted at the end of the development process, rather than a necessary process to be integrated throughout the development life cycle. Of course, this problem is not unique to ML/DL programs, but it is exacerbated when it comes to emerging technologies that do not yet have established testing methodologies. Further, the Pentagon needs to invest in and scale automated TEVV capabilities for operational platforms, such as the Navy’s automatic test and retest program,17 which will significantly speed up the testing process.

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › 1AC › 1AC - Contention Two

Cooperation on TEVV reduces the pressure for an Arms Race mentality – it builds confidence and encourages modelling.

Scharre 2021 – Director of the Technology and National Security Program at the Center for a New American Security [Paul, 6/28/21,https://tnsr.org/2021/06/debunking-the-ai-arms-race-theory/, “Debunking the AI Arms Race Theory”, 6/18/22, LND]

What can states do to avoid a race to the bottom on safety or an acceleration of the tempo of war beyond human control? In both instances, there are countervailing incentives that push against these trends. Militaries desire trusted systems on the battlefield and effective control over their own forces. There are several actions states can take to strengthen these incentives toward ensuring robust, secure, and controllable AI systems in their own institutions, as well as those of other countries. First, states should invest in adequate internal processes to test, evaluate, verify, and validate AI systems, in order to ensure that the systems they are fielding are robust and secure. States should similarly strengthen their internal processes — doctrine, training, system design and testing, human-machine interfaces, etc. — to retain effective human control over combat operations. Second, states should take specific measures to encourage other states to do likewise in order to mitigate perverse incentives to cut corners on testing or cede human control to machines where it would otherwise not be preferable. Such actions could include voluntary transparency measures about TEVV processes, although there will no doubt be technical details that states are unwilling to share. States could also communicate the importance of AI safety and reliability and of maintaining human control over combat operations, both publicly and in international diplomatic channels such as the Convention on Certain Conventional Weapons. For example, in 2020 the U.S. Department of Defense released a set of ethical principles for AI. Costly signals, such as investing in AI safety research or TEVV processes and infrastructure, may be even more effective in demonstrating to other nations that a state values fielding safe AI systems that operate under effective human control. States should avoid messages that may incentivize other states to take shortcuts on these processes, such as claims of an “AI arms race.” Lastly, states should explore opportunities to take cooperative measures that might mitigate these risks. Getting adversaries to cooperate is inherently challenging, but states have succeeded in the past in regulating the conduct of war in a variety of ways to mitigate mutual harm. Joint declarations, codes of conduct, or confidence-building measures may help to reduce the greatest dangers of AI competition and encourage states to adopt AI responsibly.

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › 1AC › 1AC - Contention Two

DOD leadership is necessary to advance TEVV – it secures funding, provides visibility, and coordinates leadership. This signals the importance of reliability and safety.

Tarraf, Shelton, and Parker 2019 - Senior Information Scientist, Senior Engineer, and physical Scientist at the RAND Corporation [ Danielle C., William, Edward, et al RAND Cooperation, “The Department of Defense Posture for Artificial Intelligence”, , Acc 6/18/22, M.A.]

DoD should advance the science and practice of VVT&E of AI systems, working in close partnership with industry and academia. The JAIC, working closely with USD(R&E), USD(A&S), and operational test and evaluation, should take the lead in coordinating this effort both internally and with external partners. VVT&E is a critical consideration for DoD (see Chapter Three and Chapter Four), a significant challenge for the entire AI community (see Appendixes B and C), and a challenge that needs to be addressed, particularly as DoD looks to employ safety-critical AI systems. VVT&E has multiple facets that need to advance: from foundational research to establish the theory and science of V&V and the theoretical underpinnings of T&E, to the development of standards, guidelines, and engineering best practices for the VVT&E of systems being fielded and operated. Because of this, multiple entities within DoD and the government have stakes in VVT&E and roles to play. Indeed, DARPA and the service labs have significant roles to play in establishing the science and its foundations, while the JAIC has a role to play in setting guidelines and institutionalizing best practices in DoD. NIST also has a role to play in developing nationwide standards. Likewise, multiple entities outside government have a stake and a role, including academics researching the science and developing the foun- 74 The Department of Defense Posture for Artificial Intelligence dations and industry seeking to continue to leverage AI at scale.9 In an ideal world, the theory and the science of VVT&E would come first. In practice, AI systems are currently being deployed, and it is therefore important to develop and institute practical alternatives, including best practices and guidelines, while waiting for the theory and science to mature.10 Leadership and open cooperation among the many stakeholders, at various levels, are required to overcome the challenge of advancing the science and practice of VVT&E. It behooves DoD to focus significant efforts and resources in this direction; to take a leadership role nationally in view of its reliance on safety-critical systems; to seek engagements and partnerships with all stakeholders in industry and academia to advance the science and practice of VVT&E; and to keep Congress informed of the status of DoD’s activities, engagements, and partnerships in this regard. The JAIC should take the lead in spearheading coordination and keeping Congress informed of this national effort, both internally and with external partners. Recommendation T-5: All funded AI efforts should include a budget for AI VVT&E. This recommendation, in support of Recommendation S-2, is a forcing function that is relatively simple to implement and that can help ensure that the consideration of VVT&E is baked into the R&D of AI techniques and the design of AI solutions rather than considered as an afterthought further down the line. Although VVT&E during early R&D phases should be commonplace, we make this recommendation to explicitly reinforce its critical importance, to highlight the present lack of foundations for VVT&E in AI, and the importance of developing that science. 9 Indeed, as we note in Chapter Three and Appendix C, the Partnership on AI—a technology-industry consortium focused on establishing best practices for AI—appears to be moving toward establishing engineering guidelines for certification of ML. 10 Some of our industry interviewees indicated taking steps, internally and in industry partnerships, toward developing best practices (see Appendix C).

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › 1AC › 1AC - Contention Two

A US commitment to cooperation on standards for AI TEVV is a confidence building measure that reduces the risk of inadvertent conflicts and builds international support for safety

Horowitz and Kahn, 2021 – Senior and Research Fellows at the Council on Foreign Relations [Michael and Lauren, The Washington Quarterly 3-19-2021 “Leading in Artificial Intelligence through Confidence Building Measures” TM]

Reducing Risks through Confidence-Building Measures Confidence-building measures could prove promising to reduce these risks of accidents, unintentional conflict, and inadvertent escalation associated with military applications of AI. The United Nations defines CBMs as “planned procedures to prevent hostilities, to avert escalation, to reduce military tension, and to build trust between countries.”16 In their simplest form, they are cross-national plans and procedures put in place to handle certain military and crisis events. Formal CBMs emerged from the 1975 Helsinki Final Act, the third meeting of the Conference on Security and Cooperation in Europe (CSCE), due to shared need and interest in preventing nuclear war.17 CBMs were specifically designed for European states’ “exchange of information, notification, and observation of major military activities,” with the aim of decreasing the risk of accidental war during the Cold War.18 Historically, CBMs work best when states fear that a lack of information could increase pressure to escalate: some mechanisms exist to mitigate lack of information; states have sufficient control over their militaries to ensure procedures are adhered to and orders are followed; and finally, when adversaries share at least some interests such as preventing accidental war. In the case of shared interests, going back to Thomas Schelling on Cold War strategic competition, cooperation between potential adversaries becomes more plausible when both sides can benefit and are incentivized to pursue the same end goal.19 We now describe four different CBMs that are applicable to the military AI arena: standardization, dialogues on AI safety, a statement on positive human control over nuclear weapons, and an autonomous incidents agreement. All focus specifically on issues surrounding accidents and miscalculation—areas where countries have shared interests in ensuring that military applications of AI do not lead to unintended conflict or escalation of tensions. Standard-Setting for Military AI, Including AI Safety Shared standards for safe uses of military AI could reduce the probability of accidents. Standards help to “herald market access, interoperability, and connectivity,” and while they might appear to be technical, standards can permit the “baking-in” of fundamental human rights protections when “ethical AI principles are translated into technical standards.”20 Furthermore, in the long run, standardization will facilitate coordination between states and ease interoperability with allied systems as well. Therefore, standard-setting for AI should start with something the United States is already committed to—following international humanitarian law. For example, including a commitment to strong TEVV requirements for AI enabled systems could further help create norms of responsible behavior surrounding the development of military uses of AI, or setting parameters on training data that can help reduce issues of bias.21 A critical element of standard setting could be commitments to follow expert-designed standards on AI safety in the TEVV process for military applications of AI.22 Key principles could include a testing framework that specifies error rates relative to existing systems, continual testing—even after deployment—rather than one-shot testing, testbeds to provide more realistic testing environments, and fail-safes built into systems in case something goes wrong. For the United States, standard-setting to enhance AI safety will only succeed if the United States does not share information that could directly help potential US adversaries. Sharing details about how the United States does TEVV, or how it is planning on doing TEVV for AI-enabled systems, could provide potential adversaries with information they would use to make their AI-enabled systems more effective. Thus, standard-setting and dialogue in these areas should focus more on themes and best practices rather than on policy details. A final element of common standards could be an emphasis on familiarizing operators with the limits of algorithms, an approach designed to reduce the potential for automation bias and even, on the other hand, automation aversion.23 An informal multilateral agreement could be proposed and opened for signature to all nations. If the United States leads, American allies and partners would be likely to sign on, both due to the impact of American leadership in shaping attitudes and the likely perception that following US-led principles would facilitate defense cooperation in this area.

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › 1AC › 1AC - Contention Two

Public commitments to cooperation on AI Testing and Evaluation promote global norms that mitigate competitive pressures that cause accidents and miscalculation

Horowitz and Scharre 2021 - Director of the Emerging Capabilities Policy Office in the Office of the Under Secretary of Defense for Policy and  Vice President and Director of Studies at CNAS [Michael, Paul, January 12 2021, “AI and International Stability: Risks and Confidence- Building Measures”, , Acc 6/18/22. M.A.]

The Limitations of AI Accident risk is a significant concern for military applications of AI. Competitive pressures could increase accident risk by creating pressures for militaries to shortcut testing and rapidly deploy new AI-enabled systems. States could take a variety of options to mitigate the risks of creating unnecessary incentives to shortcut test and evaluation, including publicly signaling the importance of T&E, increasing transparency about T&E processes, promoting international T&E standards, and sharing civilian research on AI safety. Additionally, AI will enable more capable autonomous systems. Additionally, AI will enable more capable autonomous systems, and their increased use may pose stability risks, particularly when deployed into contested areas. To mitigate these risks, states could adopt CBMs such as “rules of the road” for the behavior of autonomous systems, marking systems to signal their degree of autonomy, and adhering to off-limits geographic areas for autonomous systems. Public Signaling To reduce AI accident risk, national security leaders could publicly emphasize the importance of strong T&E requirements for military AI applications. This potentially could be linked to a formal multilateral statement or something more informal. Publicly promoting AI T&E could be valuable in signaling that nations agree, at least in principle, about the importance of T&E to avoid unnecessary accidents and mishaps. Public statements would be more powerful when used in combination with major investments in T&E institutions and processes. Promoting AI T&E as a CBM would be designed to create positive spillover effects. As major countries investing in AI come together to promote AI safety, it demonstrates the importance of the issue. It could encourage other governments to sign on and signal that AI experts within the bureaucracy can advocate for AI T&E measures.

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › 1AC › 1AC - Contention Two

NATO cooperation on TEVV is necessary for AI safety and reliability because they have specialized staff and testing centres.

Stanley-Lockman and Christie, 2021 - Innovation Officer in the Emerging Security Challenges Division in NATO and former Deputy Head of Innovation in NATO’s International Staff [Zoe and Edward, NATO Review, October “An Artificial Intelligence Strategy for NATO” , BK]

For NATO, the common commitment to these principles has practical advantages as well, providing a coherent common basis for both NATO and Allies to design and develop AI applications while also supporting interoperability goals. As such, NATO can foster the necessary interlinkages between safety, security, responsible use, and interoperability. This can be seen across the principles. For instance, it is important to ensure that AI systems are adequately robust and reliable for their intended use, not only so that they can be expected to function in accordance with legal obligations, but also to mitigate the risks of the system’s defects or limitations being exploited by nefarious actors. Putting Principles into Practice These enduring principles are also foundational to the discussion and adoption of more detailed best practices and standards. Allies and NATO can leverage NATO’s consultative mechanisms and NATO’s specialised staff and facilities to work actively towards that goal. NATO’s own standardization and certification efforts can also be bolstered by coherence with relevant international standard-setting bodies, including for civilian AI standards. In addition to best practices and standards, these principles can also be operationalized via other mechanisms including review methodologies, risk and impact assessments, and security certification requirements like threat analysis frameworks and audits, among others. Further, NATO’s cooperative activities provide the basis to test, evaluate, validate, and verify (TEVV) AI-enabled capabilities in various different contexts. More specifically, NATO’s experience not only in operations, but also in trials, exercises, and experimentation provide several avenues in which Allies and NATO can test principles against intended use cases. This is further reinforced by NATO’s scientific and technical communities, which have worked on issues such as trust, human-machine and machine-machine interactions, and human-systems integration, among many others. In addition to these existing activities, the implementation of the AI Strategy will also benefit from connections with NATO’s forthcoming Defence Innovation Accelerator for the North Atlantic (DIANA). Allied Test Centres affiliated with DIANA could be used to fulfil the aims set out in the definitions of the principles. In the future, use of these Test Centres can help ensure that AI adoption and integration are tested for robustness and resilience. For example, to ensure that AI is Traceable, Reliable and Bias-mitigating, Test Centres could synthesise how AI systems perform in different simulated environments and on different testing data, or provide independent validation and verification to assess compliance with standards that focus on responsible engineering practices.

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › 1AC › 1AC - Contention Two

The US is Key – funding of TEVV signals to allies the importance of updated AI Testing

Scharre 2021 – Director of the Technology and National Security Program at the Center for a New American Security [Paul, 6/28/21,https://tnsr.org/2021/06/debunking-the-ai-arms-race-theory/, “Debunking the AI Arms Race Theory”, 6/18/22, LND]

The United States has done more to date than any other nation to advance norms surrounding the responsible use of AI, although the Department of Defense could be more deliberate in its approach to addressing the risks of military AI competition. U.S. defense leaders have focused primarily on implementing and demonstrating AI applications in an effort to prove AI’s value in military operations. This is understandable. The Department of Defense has many practical challenges to fielding AI systems even in relatively low-risk applications, including problems with data, computing infrastructure, contracting, and funding.60 Nevertheless, it can and should do more to ensure that, as it competes in AI, it does so in a way that does not generate unnecessary risks or undermine international stability. Costly signals, such as investing in AI safety research or TEVV processes and infrastructure, may be even more effective in demonstrating to other nations that a state values fielding safe AI systems that operate under effective human control. The most important step that defense leaders could take in the near term to mitigate the risks stemming from AI competition would be to implement the necessary internal processes to ensure adequate TEVV of AI systems. A 2019 congressionally mandated independent assessment of the Defense Department’s AI efforts conducted by the RAND Corporation found that current TEVV processes were “nowhere close to ensuring the performance and safety of AI applications, particularly where safety-critical systems are concerned,” and issued recommendations for addressing this gap.61 Similarly, a 2020 independent study led by Michèle Flournoy and Avril Haines identified a number of actionable steps that the department could take to improve its AI TEVV.62 The National Security Commission on AI also concluded that “TEVV of traditional legacy systems is not sufficient” at providing adequate assurance for AI systems, and that “an entirely new type of TEVV will be needed.”63 The report issued a raft of recommendations to improve AI TEVV and establish “justified confidence in AI systems.”64 The Defense Department should adopt these reports’ recommendations to improve AI TEVV, a step that would not only bolster the safety of its AI systems but also their effectiveness. In addition to increasing resources and getting the attention of senior leadership, improving TEVV will require shifting how senior Defense Department leaders think about building robust, reliable, and effective AI systems. At times, senior defense leaders have characterized safety and ethics as an encumbrance the United States has to deal with that its adversaries do not.65 While it is undoubtedly true that Russia and China are less concerned about ethics, safety, or international law than the United States, ensuring that military AI systems operate effectively and in a way that is consistent with human intent is a strength in the long run, even if more rigorous TEVV processes are needed in the near term to achieve that goal. Another element of mitigating the risks of military AI competition is with regard to how states characterize AI. U.S. messaging has been consistent and strong on the need for the responsible, lawful, ethical, and safe use of AI.66 In their messaging, however, U.S. policymakers have frequently refrained from highlighting the risks of military AI competition, such as those outlined in this article. At times, they have emphasized a desire for speed that could feed into security dilemma concerns about a race-to-fielding that could undermine safety. In his 2020 Wired article, Will Roper wrote: “Our nation must wake up fast. The only thing worse than fearing AI itself is fearing not having it.”67 While he called for using AI “safely and effectively,” his overriding emphasis was for the Defense Department to move faster.68 Quite understandably, U.S. policymakers working to accelerate the adoption of AI in a slow-moving and sclerotic bureaucracy may be loath to portray the technology as immature, unready, or unreliable. Additionally, U.S. policymakers may fear that highlighting the risks of military AI could contribute to AI engineers balking at working with the military, even if such fears are unfounded.69 Nevertheless, just as other emerging technologies such as computer networks opened up novel strategic challenges in the form of cyber operations, defense analysts should begin to think now about ways that AI may complicate international stability. A forthright acknowledgment of the risks of military AI competition is the first step toward a strategy of competing smartly while addressing those risks. Acknowledging the risks of military AI competition does not mean that the United States should refrain from adopting AI any more than acknowledging the stability risks of competing in space, cyberspace, or nuclear weapons necessitates unilateral disarmament in those spheres. America’s response to these risks should not be to refrain from AI competition, but rather to shape the character of the competition so that states are, at a minimum, mindful of these risks.

2022 MNDI Affirmative
TEVV AFF – MNDI HPA › 1AC › 1AC - Contention Two

Expanding TEVV prevents AI accidents – an Arms Race dilemma rushes deployment of unreliable weapons, which undermines effectiveness and safety.

Scharre 2021 – Director of the Technology and National Security Program at the Center for a New American Security [Paul, 6/28/21,https://tnsr.org/2021/06/debunking-the-ai-arms-race-theory/, “Debunking the AI Arms Race Theory”, 6/18/22, LND]

A related risk of a “racing” dynamic among competitors could come from an acceleration, not of the pace of operations on the battlefield, but of the process of fielding new AI systems. AI systems today have a host of safety and security problems that can make them brittle, unreliable, and insecure.29 Because machine learning in particular can create new ways in which systems can fail, militaries face novel challenges in adopting AI systems.30 Militaries will have to adopt new methods to test, evaluate, verify, and validate AI systems (also known as TEVV).31 Such concerns related to autonomy are well known in the U.S. defense community,32 although at present they have not been solved to a satisfactory degree. Machine learning introduces additional challenges with regard to testing, evaluation, verification, and validation. A rush to field AI systems before they are fully tested could result in a “race to the bottom” on safety, with militaries fielding accident-prone AI systems. There are strong bureaucratic and institutional imperatives for militaries to field systems that are robust and secure. Indeed, designing systems to military specification standards often means making them more robust for a wider range of environmental conditions and shocks than comparable commercial systems, even at the expense of other aspects of performance, such as size, weight, or usability. AI presents novel challenges, however, in achieving the robustness needed for operating in the complex, hazardous, and adversarial environments that often characterize military operations. Certain AI methods today, such as deep learning, remain relatively immature with significant reliability challenges. A 2017 Department of Defense report by the JASON scientific advisory group explained that deep neural networks are immature as regards the “illities”, including reliability, maintainability, accountability, validation and verification, debug-ability, evolvability, fragility, attackability, and so forth. … Further, it is not clear that the existing AI paradigm is immediately amenable to any sort of software engineering validation and verification. This is a serious issue, and is a potential roadblock to DoD’s [Department of Defense’s] use of these modern AI systems, especially when considering the liability and accountability of using AI in lethal systems.33 The Defense Department’s 2018 AI strategy calls for building AI systems that are “resilient, robust, reliable, and secure.”34 Yet, the current state of technology makes achieving this goal particularly difficult for AI systems that incorporate deep learning, a subfield of AI that has seen significant growth and attention in recent years. While there is active research underway to improve AI safety and security, militaries will have to adapt to the technology as it currently is, at least for the time being. An ideal process would be for militaries to engage in experimentation, prototyping, and concept development, but also to subject AI systems to rigorous TEVV under realistic operational conditions before deployment. Taking shortcuts on testing and evaluation and fielding a system before it is fully tested could lead to accidents, which, in some settings, could undermine international stability. In evaluating new technologies, militaries may be relatively accepting of the risk of accidents, which may lead them to tolerate the deployment of systems that have reliability concerns. In building and fielding new capabilities, militaries have to weigh the possibility of an accident occurring against other concerns, such as forgoing valuable military capabilities. The military operational environment is fraught with risk, in both training and real-world operations. Military institutions balance managing this risk with other factors, such as the need for training, developing new capabilities, or accomplishing the mission. Military institutions view casualties from training accidents or testing new capabilities as a tragic but unavoidable part of the business of preparing for war. Militaries expect high performance from their forces, often while they are performing dangerous tasks, but militaries neither demand nor expect accident-free operations in most settings.35 From 2006 to 2020, over 5,000 U.S. servicemembers were killed in non-war related accidents, the majority of which occurred within the United States. Accidents overall accounted for nearly 32 percent of U.S. servicemember deaths during this period, and even accounted for a significant portion of servicemember deaths in Iraq (19 percent) and Afghanistan (16 percent).36 These accident rates are not unusual for the U.S. armed forces. This is business as usual. Accidents draw the attention of senior military and civilian officials when a spate of accidents occur in a short amount of time — such as a series of aircraft crashes,37 ship collisions,38 or training accidents.39 Yet, as one report on naval accidents from 1945 to 1988 notes, “peacetime naval accidents are a fact of life.”40 The same is true of military air and ground operations. Other nations’ militaries may do an even poorer job of managing risk when it comes to accidents than the U.S. military. For example, the Soviet/Russian submarine community has a much higher accident rate than the U.S. submarine community.41 New technologies in particular present an increased risk of accidents, yet militaries may press ahead out of a desire to develop and field what they perceive to be a valuable capability. For example, the V-22 Osprey tiltrotor aircraft suffered four crashes during development, killing 30 U.S. servicemembers in total, yet the Defense Department continued development.42 The V-22 program manager cited a rush to develop the technology as a factor in the accidents, stating, “Meeting a funding deadline was more important than making sure we’d done all the testing we could.”43 Taking shortcuts on testing in particular appears to have been a factor in at least one fatal crash. According to a Government Accountability Office investigation of the V-22 program, “schedule pressures” led the program to conduct only 33 of 103 planned tests of an aerodynamic phenomenon called a “vortex ring state,”44 a phenomenon that later caused an April 2000 crash that killed 19 servicemembers.45 Absent competitive dynamics, militaries may be able to manage the challenges of fielding safe AI systems to a more-or-less satisfactory degree, albeit with some risk of an accident occurring. However, out of a desire to field AI capabilities ahead of competitors, militaries may be more willing to accept risk than they might otherwise be and to field systems that are prone to mishaps.46 Similar competitive dynamics may have played a role in accidents with self-driving cars and commercial airline autopilot technology, as companies rushed to beat others to market.47 These dynamics, while not an arms race, could lead militaries to engage in a “race to the bottom” on safety. This risk could become particularly acute in wartime. New technologies in particular present an increased risk of accidents, yet militaries may press ahead out of a desire to develop and field what they perceive to be a valuable capability. Managing these risks is challenging because assessing them can be difficult, especially when it comes to new technologies. Accident rates may be well-known for mature technologies, but they are unknown for technologies still in development. In the case of the V-22 Osprey development, for example, it is not as though the Defense Department knew that developing it would lead to multiple crashes and 30 fatalities but decided that achieving the capability was worth the cost. Engineers, testers, and program managers are flying in the dark when it comes to new technologies — that is, after all, the point of testing new systems. The concern is not only that organizations may take measured risks to field new capabilities, but also that institutional and bureaucratic imperatives may lead organizations to distort their own perceptions of risk, further contributing to accidents. This sociological phenomenon has been cited as a cause in the 1986 Space Shuttle Challenger explosion, for example.48