TEVV is crucial for Operator Trust in autonomous systems
Haugh, 2018 - project leader for the Institute for Defense Analysis [Brian A., September “The Status of Test, Evaluation, Verification, and Validation (TEV&V) of Autonomous Systems,” https://www.ida.org/-/media/feature/publications/t/th/the-status-of-test-evaluation-verification-and-validation-of-autonomous-systems/p-9292.ashx, 6/23/22 MD]
4. CONOPS and training as design features To date, the paradigm for designing systems has been to make a reasonable guess about how the operator(s) will use that system and what the user interface should look like, and then work out the details of the Concept of Operations (CONOPS), tactics for effective employment, and training of future operators long after the basic design has been decided. For autonomous systems, where the system operates itself to some extent and interacts autonomously with humans, the details of CONOPS and tactics (and corresponding training) are part of the system design, at least on the machine side (and probably on the human side as well), and will have to be identified, verified, and validated much earlier in the development process. This will pose organizational and personnel challenges to T&E, as well as methodological challenges. 5. Human trust In human-machine teaming (HMT) contexts, how the humans behave (and thus how well the team performs) depends in part on the humans’ psychological attitudes toward the autonomous systems. “Trust” is the term generally used to describe those attitudes, though in practice those attitudes are generally more nuanced and multi-dimensional than simply asking “how much do I trust it?” In order to design, debug, and assure performance, TEV&V will need to be able to measure the various dimensions of trust, to support understanding of how trust affects team performance.