Neural Networks for AI increase mistakes – the “Black Box” makes it impossible for researchers to examine AI decisions.
Scharre, 2016 -- Vice President and Director of Studies at CNAS [Paul, Feb 2016, Center for New American Security, “Autonomous weapons and operational risk”, https://s3.us-east-1.amazonaws.com/files.cnas.org/documents/CNAS_Autonomous-weapons-operational-risk.pdf?mtime=20160906080515&focal=none, Acc 6/21/22, M. A.]
Neural networks and the black box The challenge of complexity, while problematic for complex rule-based systems, is even more difficult for cutting-edge artificial intelligence (AI) systems that employ neural networks. Neural networks do not perform rule-based calculations like most computers. Instead, they learn by exposure to large data sets. As a result, the internal structure of the network that generates output can be opaque to the designers—a “black box.” Even more unsettling, for reasons that may not be entirely clear to AI researchers, the neural network sometimes can yield odd, counterintuitive results. A study of visual classification AIs using neural networks found that while the AIs were able to generally identify objects as well as humans, in some cases the AIs made confident identifications of objects that were not only incorrect, but that looked vastly different from the purported object to human eyes. The AIs interpreted images that to the human eye looked like static or abstract wavy lines as animals or other objects, and asserted greater than 99.6% confidence in their estimation.