Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Untested AI systems jeopardize nuclear command and control – they are susceptible to hacking by non-state actors

Fitzpatrick, 2019 - former the IISS Non-Proliferation and Nuclear Policy Programme [Mark, 21 May, “Artificial Intelligence and Nuclear Command and Control,” https://www.iiss.org/blogs/survival-blog/2019/04/artificial-intelligence-nuclear-strategic-stability 6/18/22 MD]

Among the most revealing vulnerabilities exposed in the exercise was the way in which private-sector technological breakthroughs might be expropriated by another non-state actor for malevolent purposes, with dire consequences for strategic stability. In the hypothetical two months before the exercise started, a US technology company called QuantumAI, which received significant seed funding from the CIA and won Defense Advanced Research Projects Agency contracts for its advanced magnetometers and gravimeters, partnered with another company to launch four open-source, quantum-sensing satellites that used AI to measure infinitesimal changes in magnetic and gravitational fields on the earth. Subscribers began using the new system to identify new mining opportunities, sunken shipwrecks and toxic metallic effluents from industrial sites. QuantumAI only marketed its product to government-approved researchers from NATO countries, and its technology was export-controlled under both US International Traffic in Arms Regulations restrictions and the Wassenaar Arrangement. In round one, satellite imagery of the South China Sea that rendered the water transparent and revealed the location of three Chinese submarines was put online, allegedly from sources linked to QuantumAI. While the Chinese team suspected that the US government was responsible, the US team worried that whoever put this information online might similarly reveal the location of US submarines. Exactly that transpired in round two. The same website posting locational data regarding Chinese submarines expanded its coverage to include a global map with additional markers for Russian and US nuclear submarines. This posed a clear risk to strategic stability in compromising the second-strike capability that stealthy nuclear-armed submarines provide. Adding the markers for US submarines made it clear that the revelatory website was not affiliated with the US government. Its registration history and hosting service provider in Switzerland suggested it was actually linked to the World Peace Guardians. When US Department of Homeland Security and FBI officials approached large US social-media firms for information regarding World Peace Guardians accounts, the companies replied that they would ‘share relevant information with the government when it becomes available’. The US president did decide that QuantumAI hard drives could be seized, although the CEO of QuantumAI was not fully cooperative. The idea of trying to seize World Peace Guardians servers in Switzerland was discussed, but rejected. The QuantumAI aspect of the exercise was one manifestation of what the background paper described as a ‘black box model extraction’ vulnerability. Such an extraction reverse-engineers an AI system to determine its parameters. An adversary may be able to use this information to enhance the effectiveness of future operations against the system by stealing intellectual property; identifying sensitive or proprietary information related to the system’s training data or objectives; or developing ‘adversarial inputs’ to be covertly introduced into the original AI system. Such inputs confuse the system’s classifiers or its pattern-recognition function, thereby causing it to miscalculate, misclassify or misinterpret elements in its operational environment.

Sign in to recut, update, or save this card.