Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Human control cannot solve for instability – adversaries can never know if the system’s behavior followed human intentions or AI

Horowitz and Scharre 2021 - Director of the Emerging Capabilities Policy Office in the Office of the Under Secretary of Defense for Policy and Vice President and Director of Studies at CNAS [Michael, Paul, January 12 2021, “AI and International Stability: Risks and Confidence- Building Measures”, https://www.cnas.org/publications/reports/ai-and-international-stability-risks-and-confidence-building-measures, Acc 6/18/22. M.A.]

Finally, autonomous systems raise novel challenges of signaling in contested areas because of ambiguity about whether their behavior was intended by human commanders. Even if the system performs as intended, adversaries may not know whether an autonomous system’s behavior was consistent with human intent because of the aforementioned command-and-control issues. This can create ambiguity in a crisis situation about how to interpret an autonomous system’s behavior. For example, if an autonomous system fired on a country’s forces, should that be interpreted as an intentional signal by the commanding nation’s political leaders, or an accident? This, again, is not a novel problem; a similar challenge exists with human-commanded military forces. Nations may not know whether the actions of an adversary’s deployed forces are fully in line with their political leadership’s guidance. Autonomous systems could complicate this dynamic due to uncertainty about whether the actions of an autonomous system are consistent with any human’s intended action.

Sign in to recut, update, or save this card.