Turn – Operator Trust – ethical use policies increase a soldier’s Trust in their weapons, which is essential to their effective use. It doesn’t matter how good your AI weapons are if the military won’t use them.
Gilli, 2020 - Senior Researcher at the NATO Defense College [Andrea, NDC Research Paper No.15 – December ““NATO-Mation”: Strategies for Leading in the Age of Artificial Intelligence” https://www.ndc.nato.int/news/news.php?icode=1514 Acc 4/21/22 TA]
This is important because AI is data-intensive, unpredictable and brittle.98 While a system may work well in the context in which it was trained, it may break in an unfamiliar setting. The reliance on BD in the current second wave of ML also creates fallibilities,99 given that the algorithms can scale up harm if the data over- or under-represents certain groups.100 For neural networks in particular, it may be impossible to explain or interpret results. Some refer to this as the “black-box” problem, meaning that the outcomes of these complex AI systems are opaque to humans that either want to reproduce the good, or prevent the bad from recurring. While traditional software can be debugged to solve a performance issue, the lack of linear causality between a programmer’s inputs and the AI system’s outputs means that it is difficult to track bias and reliability. Creating organizational processes to minimize these concerns across the AI lifecycle is critical to responsible use of the technology. Considering AI ethics also means developing trust in systems. Organizations developing AI applications should think deeply about user expectations related to transparency and disclosure. Ultimately, these norms will change the distinction between human, AI-assisted and AI interactions.101 “Calibrating” trust is especially important for AI-assisted decision making,102 a concept whose value will only increase given the emphasis on human-machine teaming in Allied militaries.103 In many cases, the unethical or unacceptable outcomes of AI systems pertain not to moral dilemmas, but to the reliability and robustness of the systems at hand. As such, building trust is fundamentally tied to building safe and secure systems.104 The problem of bias in AI illustrates the overlap between AI safety and AI ethics: bias is morally problematic, because it can unfairly harm or systematically discriminate against specific groups105 – and it can also be seen as a failure mode that reduces the reliability of a given algorithm in a given context.106 For NATO, this relates to the safety of enterprise tools and weapons systems alike. In deployments, these safety measures would be critical to ensure that AI-enabled weapons systems are used in a manner consistent with the principles of international humanitarian law. As explained in greater detail below, this may also feed into the eventual standardization process.