Testing and Reliability are essential to prevent cyber attacks
National Security Commission on Artificial Intelligence 2022 [March 19, “NCSAI Final Report 2021 – Chapter 7: Establishing Justified Confidence in AI Systems”, https://assets.foleon.com/eu-west-2/uploads-7e3kk3/48187/nscai_ch7_digital_02-26-21.24159789cf2e.pdf]
1. Robust and Reliable AI. Current AI systems, such as those used for perception and classification, have different kinds of failure––characterized as rates of false positives and false negatives. They are often brittle when operating at the edges of their performance competence, and it is difficult to anticipate their competence boundaries.3 They are also vulnerable to attack, and they can exhibit unwanted bias in operation. For national security missions, these can be serious problems. U.S. government agencies should: Focus more federal R&D investments on advancing AI security and robustness. These investments should also advance the interpretability and explainability of AI systems, so users can better understand whether the systems are operating as intended. Consult interdisciplinary groups of experts to conduct risk assessments, improve documentation practices, and build overall system architectures to limit the consequences of system failure.4 Such architectures should securely monitor component performance and handle errors when anomalies are detected5 ; contain AI components that are self-protecting (validating input data) and self-checking (validating data passed to the rest of the system); and include aggressive stress testing.