Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

This turns the case – the security dilemma pressures states to push out AI before it is safe, which causes it to fail, risking unintentional escalation of conflicts

Horowitz and Kahn, 2021 – Senior and Research Fellows at the Council on Foreign Relations [Michael and Lauren, The Washington Quarterly 3-19-2021 “Leading in Artificial Intelligence through Confidence Building Measures” https://www-tandfonline-com.proxy.lib.umich.edu/doi/full/10.1080/0163660X.2021.2018794 acc on 6-21-2022 TM]

Risks exist for AI uses across society, in civilian and military sectors alike.7 Survey research shows that many in AI and machine learning (ML) communities worry about whether militaries can be trusted to develop safe and reliable AI.8 Given the current geopolitical context, there is fear that competitive pressures to be first in deploying AI systems could overshadow safety and ethics efforts, incentivizing militaries to take “short cuts” that could increase the risk of lethal accidents, unintentional conflict (conflict between states that results from miscommunication or accidents), and inadvertent escalation (when states commit intentional actions that unintentionally cause escalation by an adversary). The Risk of Accidents First, military applications of AI may not work as intended due to flaws in the algorithms themselves. Inadequate training data, the algorithm’s complexity, biased data, biased coding, or intentional data sabotage or poisoning by adversaries can make accidents more likely when systems are deployed. For example, a malfunctioning AI-enabled targeting system in a conflict could attack friends instead of foes, or friends and foes alike. Or an algorithmic decision aid could derive a flawed estimate of the risk of a particular operational plan, leading to a use of force that either unnecessarily fails or succeeds at a greater cost than required. These risks are amplified when systems are employed outside their design context, which may be more likely due to competitive dynamics.9 The Risk of Unintentional Conflict Second, even if algorithms work as designed, they could unintentionally make conflict more likely. Uncertainty about how algorithms will work on the battlefield could create challenges for signaling adversaries through posture and deployments in a crisis or within a conflict. Actors might be uncertain about how A Ienabled systems deployed by an adversary will behave. They might not believe that AI-enabled autonomous systems are programmed as described because the black-box nature of algorithms could lead to increased skepticism of claims made about the systems. Even if a state is telling the “truth” about a specific system and how it was used, there might not be a way for a third party or another state to validate those claims. Mistrust due to great power competition could exacerbate the situation. There is still substantial uncertainty about the reliability of many AI methods, especially deep learning and related techniques when they generate opaque outputs, meaning there is not an available chain of logic which explains why the algorithm recommended or engaged in a particular action.10 Even with sufficient training data, hedges against biases, and protections from data poisoning and hacking, current DoD systems may not be prepared for testing, evaluation, validation, and verification (TEVV) of algorithmic systems, particularly AI-enabled autonomous systems that will continually learn while operating.11 Furthermore, in operating at machine speed, AI-enabled autonomous systems could cause adversaries in a conflict to fear losing a war so quickly that they feel compelled to escalate.12 For an adversary with nuclear weapons, this could create pressure for launch postures such as pre-delegation and launch on warning that increase the chance of nuclear use.

Sign in to recut, update, or save this card.