Taddeo & Floridi 18
NATURE | VOL 556 | 19 APRIL 2018, Mariarosaria Taddeo is a research fellow and deputy director of the Digital Ethics Lab at the Oxford Internet Institute, University of Oxford, UK; and a Turing fellow of the Alan Turing Institute, London, UK. Luciano Floridi is professor of philosophy and ethics of information at the University of Oxford, UK; director of the Digital Ethics Lab at the Oxford Internet Institute; and chair of the Data Ethics Group at the Alan Turing Institute. “Regulate artificial intelligence to avert cyber arms race”, https://media.nature.com/original/magazine-assets/d41586-018-04602-6/d41586-018-04602-6.pdfv --ECM
International dialogue and action must resume. NATO could pave the way through its forthcoming guidelines, although it is currently unclear what their scope will be.
Meanwhile, research on AI for cyberdefence is progressing quickly. The United States is in the lead, technologically. It aims to incorporate AI into its cyberdefence systems by 2019 (ref. 3). The US Department of Defense (DOD) has earmarked $150 million for research. The US Defense Advanced Research Projects Agency (DARPA) is developing the techniques and strategies. Steps have already been taken. In DARPA’s 2016 Cyber Grand Challenge competition, seven AI systems, developed by teams from the United States and Switzerland, fought against each other. The systems identified and targeted their opponents’ weaknesses while finding and patching their own.
The DOD will issue the first US report on AI strategies for national defence in May. There is, as far as we know, no indication of what its approach will be. Previous documents, such as The DOD Cyber Strategy from 2015 or the 2016 National Cyber Incident Response Plan, did not cover autonomous systems, machine learning or AI. The 2012 DOD directive on ‘Autonomy in Weapon Systems’ focused on internal procedures for deploying AI but was silent on when the United States would do so in the international arena.
AI is a priority for China, which aims to become a world leader in machine-learning technologies. In July 2017, the Chinese government issued its Next Generation AI Development Plan. Military implementation of AI, on the battlefield as well as in cyberspace, is a crucial part of the strategy. But it is unclear to what degree China plans to deploy AI actively in cyberdefence.
Russia has not released any public documents about its strategies for AI in defence. However, in a video message released in 2017, President Vladimir Putin referred to AI and stated: “Whoever becomes the leader in this sphere will become the ruler of the world.” Experts agree that Russia is focusing on developing AI-enhanced tools for its conventional forces. However, since 2014, the Russian National Defense Control Center has been using machine-learning algorithms to detect online threats. Allegedly, Russia has pioneered the use of AI to spread disinformation and intervene in the public debates of other nations, including the 2016 US presidential election and the United Kingdom’s EU membership referendum. Although these operations are not part of national defence strategies, they indicate Russia’s advanced AI capabilities.
North Korea has a history of cyberspace aggression. It was implicated, for example, in the WannaCry attack in 2016 and in another major breach, against Sony Pictures, in 2014. The country lacks technical expertise in AI but is likely to want to catch up with its adversaries.
The EU is stepping up, too. In 2017, it reassessed cybersecurity and defence policies and launched the European Centre of Excellence for Countering Hybrid Threats, based in Helsinki. The EU has the most comprehensive regulatory framework for state conduct in cyberspace so far. Yet these directives do not go far enough. The EU treats cyberdefence as a case of cybersecurity, to be improved passively by making member states’ information systems more resilient. It disregards active uses of cyberdefence and does not include AI.
This is a missed opportunity. The EU could have begun defining red lines and proportionate responses in its latest rethink. For example, the 2016 EU directive on ‘Security of Network and Information Systems’ provides criteria for identifying crucial national infrastructures, such as health systems or key energy and water supplies that should be protected. The same criteria could be used to define illegitimate targets of state-sponsored cyberattacks.
Regional forums, such as NATO and the EU, must take the following three steps to avoid serious imminent attacks on state infrastructures, and to maintain international stability.
THREE STEPS
Define legal boundaries. The international community needs to agree urgently on red lines that distinguish between legitimate and illegitimate targets. Also needed are definitions of proportionate responses for cyberdefence strategies. International consensus at the UN level will ultimately be required. Until then, guidelines from regional multilateral bodies, such as NATO and the EU, must cover these issues and lead by example.
Test strategies with allies. ‘Sparring’ exercises should be organized between friendly countries to test AI-based defence tactics. These tests should be mandatory before any system is deployed. They could be in the form of DARPA’s Grand Challenge or the simulation exercises routinely run by NATO and the EU. Because AI learns by experience, these matches will improve the strategies of the alliance, while finding and healing weaknesses. Fatal vulnerabilities of key systems and crucial infrastructures should be shared with allies; policy frameworks should demand disclosure. Agreements and regulations with similar sharing and disclosure requirements include the EU Electronic Identification, Authentication and Trust Services Regulation and NATO’s Industry Partnership Agreement.
Monitor and enforce rules. The international community needs to agree how to audit and oversee AI-based state cyberdefence operations. ‘Alert and remedy’ mechanisms are needed to address mistakes and unintended consequences. A third-party authority with teeth, such as the UN Security Council, should rule on whether red lines, proportionality, responsible deployment or disclosure norms have been breached. Economic or political sanctions should be imposed on states that violate rules. NATO and the EU should enforce the norms within their remits.
The solution is difficult, but it is clear. There is no time to waste.