“Arms Race” mentality pressures rapid deployment of weapons – militaries skip critical testing and evaluation.
Horowitz and Scharre 2021 - Director of the Emerging Capabilities Policy Office in the Office of the Under Secretary of Defense for Policy and Vice President and Director of Studies at CNAS [Michael, Paul, January 12 2021, “AI and International Stability: Risks and Confidence- Building Measures”, https://www.cnas.org/publications/reports/ai-and-international-stability-risks-and-confidence-building-measures, Acc 6/18/22. M.A.]
An additional challenge stems from security dilemma dynamics. Competitive pressures could lead nations to shortcut test and evaluation (T&E) in a desire to field new AI capabilities ahead of adversaries. Similar competitive pressures to beat others to market appear to have played an exacerbating role in accident risk relating to AI systems in self-driving cars and commercial airplane autopilots. Militaries evaluating an AI system of uncertain reliability could, not unjustifiably, feel pressure to hasten deployment if they believe others are taking similar measures. Historically, these pressures are highest immediately before and during wars, where the risk/reward equation surrounding new technologies can shift due to the very real lives on the line. For example, competitive pressures may have spurred the faster introduction of poison gas in World War I. Similarly, in World War II, Germany diverted funds from proven technologies into jet engines, ballistic missiles, and helicopters, even though none of the technologies proved mature until after the war. This dynamic risk might spark a self-fulfilling prophecy in which countries accelerate deployment of insufficiently tested AI systems out of the fear that others will deploy first. The net effect is not an arms race but a “race to the bottom” on safety, leading to the deployment of unsafe AI systems and heightening the risk of accidents and instability.