Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

TEVV is key to recognize new cyber attacks on autonomous AI

Haugh, 2018 - project leader for the Institute for Defense Analysis [Brian A., September “The Status of Test, Evaluation, Verification, and Validation (TEV&V) of Autonomous Systems,” https://www.ida.org/-/media/feature/publications/t/th/the-status-of-test-evaluation-verification-and-validation-of-autonomous-systems/p-9292.ashx, 6/23/22 MD]

7. Exploitable vulnerabilities When systems operate themselves, they can be vulnerable to modes of attack— cyber, electronic, or physical—that would not be as much of a concern for a human-operated system. For example, a cyberattack that compromised the ability of an autonomous unmanned aircraft system (UAS) to recognize other aircraft or physical proximity illusions that repeatedly triggered the UAS’s collision avoidance subroutine might be much more effective than they would be against a human-piloted aircraft. AI based on machine learning has its own set of potential vulnerabilities, both during training of the AI and in operation. TEV&V of autonomous systems will need to be aware of this expanded attack surface. 8. Emergent behavior US DoD Directive 3000.09 specifically warns against the possibility of “unanticipated emergent behavior resulting from the effects of complex operational environments on autonomous or semi-autonomous systems.” Developing T&E methods to analyze the potential for emergent behavior in order to avoid it will be central to providing assured dependability for autonomous systems.

Sign in to recut, update, or save this card.