Autonomous Incidents Agreements fail – Incentives for breaking and untruthfulness outweigh agreement
Horowitz and Scharre 2021 - Director of the Emerging Capabilities Policy Office in the Office of the Under Secretary of Defense for Policy and Vice President and Director of Studies at CNAS [Michael, Paul, January 12 2021, “AI and International Stability: Risks and Confidence- Building Measures”, https://www.cnas.org/publications/reports/ai-and-international-stability-risks-and-confidence-building-measures, Acc 6/18/22. M.A.]
One challenge to establishing rules of the road for autonomous systems’ behavior would be if there were incentives to defect from the rules. For example, in World War I, technological developments enabled submarines, which were highly effective in attacking ships but unable to feasibly comply with existing prize law without putting themselves at risk of attack by surfacing. Despite attempts in the early 20th century to regulate submarines, the incentives for defecting from the existing rules were too great (and the rules failed to adapt), and the result was unrestricted submarine warfare. Another challenge to a potential autonomous incidents agreement is fully exploring the incentives for trustworthiness, both in the signals that countries send about the behavior of their autonomous systems and adversaries’ responses. Some declaratory policies would not be credible, such as the claim to have created a “dead hand” system such that if a country engaged in a particular type of action, an autonomous system would start a war and there would be nothing a leader could do to stop it.