Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Increasing training and refining AI issues to incorporate more human controls decreases errors and increases trust – CBMs work

Atherton 2022 – Military Technology Journalist [Kelsey, 5/6/22, “Understanding the errors introduced by military AI applications”, https://www.brookings.edu/techstream/understanding-the-errors-introduced-by-military-ai-applications/, 6/18/22, LND]

Finding the right mix of trust between an autonomous machine and the human relying on it is a delicate balance, especially given the inevitability of error. Seventeen years after the Tornado shootdown, the automated features of the Patriot missile remain in place, but the way in which they are used has shifted. Air threats, such as aircraft, helicopters, and cruise missiles can now only be engaged in manual mode “to reduce the risk of fratricide,” as the U.S. Army’s manual for air and missile defense . In manual mode, automated systems still detect and track targets, but it’s a human who makes the call about when and if to fire. But “for ballistic missiles and anti-radiation missiles,” like the kind the Patriot in Iraq assumed the Tornado was, “the operator has a choice of engaging in the automatic or manual mode,” though the manual notes that these “engagements are typically conducted in the automatic mode.” Defense researchers caution that human beings are not well-suited to monitoring autonomous systems in this way. “Problems can arise when the automated control system has been developed because it presumably can do the job better than a human operator, but the operator is left in to ‘monitor’ that the automated system is performing correctly and intervene when it is not,” the engineering psychologist John Hawley, who was involved in the U.S. Army’s efforts to study the 2003 friendly fire incidents, in a 2017 report. “Humans are very poor at meeting the monitoring and intervention demands imposed by supervisory control.” 

Sign in to recut, update, or save this card.