Testing is key---establishes standards for interoperability and data-sharing
Andrew Imbrie et al 20, Andrew Imbrie Ryan Fedasiuk Catherine Aiken Tarun Chhabra Husanjot Chahal. Center for Security and Emerging Technology (CSET) at Georgetown’s Walsh School of Foreign Service is a research organization focused on studying the security impacts of emerging technologies, supporting academic work in security and technology studies, and delivering nonpartisan analysis to the policy community. February. "Agile Alliances: How the United States and its Allies Can Deliver a Democratic Way of AI" //pipk
Interoperability is a critical lubricant for U.S. alliances. To operate effectively, allies need to plan, train, and exercise together. Joint operational concepts, common doctrine, and compatible military capabilities and systems are required to communicate effectively and achieve shared objectives.66 As countries integrate AI into military systems, the United States and its allies must ensure that hardware and digital systems are interoperable and secure.
The United States and its allies could start with common standards for interpretability, safety, and security of AI systems, including AI-enabled, safety-critical systems.67 For AI-enabled military systems expected to perform a given function, the United States and its allies should agree on common benchmarks for accuracy and performance based on the same training and testing data. The CSET survey suggests that allies and partners desire such benchmarks, with a majority of surveyed officials expressing the need for international coordination and management of AI military applications, specifically autonomous weapons systems and unmanned vehicles for submarine detection. A German representative stated that collaboration with the United States would be enhanced by an AI strategy that includes a focus on AI-related defense and security threats.
The United States and its allies should also consider wargaming and table-top exercises to explore how sharing selected government data sets could shore up defenses against counter-AI techniques and other efforts to exploit the vulnerabilities of AI systems. Specifically, they should explore how sharing militarily relevant data sets and certain AI algorithms could help allied countries better test system robustness, expose mutual vulnerabilities, accelerate development of countermeasures, and establish common standards for testing, verification, and validation.68
The United States and its allies should define common standards for the level of robustness required for a given operation. Common defense planning and capability development in NATO and the EU should give priority to investments in AI safety and security, as well as common verification procedures for AI-enabled, safety-critical systems.