Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Adding Run Time Monitoring improves traditional TEVV by extending testing beyond simulations

Haugh, 2018 - project leader for the Institute for Defense Analysis [Brian A., September “The Status of Test, Evaluation, Verification, and Validation (TEV&V) of Autonomous Systems,” https://www.ida.org/-/media/feature/publications/t/th/the-status-of-test-evaluation-verification-and-validation-of-autonomous-systems/p-9292.ashx, 6/23/22 MD]

D. Run-Time Monitoring Given the difficulty of assuring that a system will not exhibit specific undesired behaviors, a natural thought is to instead monitor the system during operations and intervene when bad behavior is imminent. This approach is already common in engineering practice for safety-critical systems. It is mentioned explicitly in the recent US Unmanned Systems Integrated Roadmap 2017-2042: For the most demanding adaptive and non-deterministic systems, a new approach to traditional TEVV will be needed. For these types of highly complex autonomous systems, an alternate method leveraging a run-time architecture that can constrain the system to a set of allowable, predictable, and recoverable behaviors should be integrated early into the development process. Emergent behaviors from large-scale deployment of interacting autonomous systems poses a difficult challenge. The analysis and test burden would thereby, be shifted to a simpler, more deterministic run-time assurance mechanism. The effort for new approaches to TEVV endeavors to provide a structured argument, supported by evidence, justifying that a 3-6 system is acceptably safe and secure not only through offline tests, but also through reliance on real-time monitoring, prediction, and fail-safe recovery.1 Although this mechanism might indeed be simpler than avoiding unpredictable behaviors in the first place, it is not without its own challenges. In general, any behavior whose dependability cannot be adequately assured through system design and training would need to be monitored, with a robust intervention standing by. This means not only intervening when the system is about to execute some undesired physical action (e.g., one that might risk harm to the system or to humans), but also intervening in any case where the system is making a bad decision or misinterpreting its environment. Detecting such cases and handling them gracefully will not always be easy. Research is required into architectures to support this concept, instrumentation needs and control algorithms to predict and avoid specific failure modes, systematic identification of conditions to be monitored for, robustness against attacks designed to invoke fail-safe behaviors, and so forth. It goes without saying that the fail-safe systems would themselves need to be verified and validated as well.

Sign in to recut, update, or save this card.