Testing and Verification are essential to AI cyber security – transparency is key to effective Deterrence of cyber threats
Taddeo and Floridi, 2018 - Senior research fellow at the Oxford Internet Institute and Professor of Philosophy and Ethics of Information at the University of Oxford [Mariarosaria, Luciano, Nature.com, “Regulate artificial intelligence to avert cyber arms race,” https://www.nature.com/articles/d41586-018-04602-6 6/18/22 MD]
Cyber-attacks are becoming more frequent, sophisticated, and destructive. Each day the US suffers more than 4,000 ransomware attacks that encrypt computer files until the owner pays to release them [1]. This is triple the number in 2015. In May 2017, when WannaCry virus crippled hundreds of IT systems across the UK’s National Health Service, 20,000 appointments were cancelled. A month later, the NotPetya ransomware cost pharmaceutical giant Merck, shipping firm Maersk, and logistics company FedEx around $300 million each in immediate damages. Global damages from cyber-attacks totalled $5 billion in 2017 and may reach $6 trillion a year by 2021. States are partly behind this rise. They use cyber-attacks both offensively and defensively. For example, North Korea has been linked to WannaCry, Russia to NotPetya. As the threats escalate, so do defence strategies. Since 2012, the US has employed ‘active’ cyber defence strategies that enable computer experts to neutralise or distract viruses with decoy targets, and to break back into a hacker’s system to delete data or to destroy it completely. In 2016, the UK announced a £1.9bn investment and a 5-year plan to combat cyber threats. NATO also began drafting principles for such operations, to be agreed by 2019. The US and the UK are leading this initiative, with Denmark, Germany, the Netherlands, Norway, and Spain also involved [2]. Artificial intelligence (AI) is poised to revolutionize this activity. Attacks and responses will become faster, more precise, and more disruptive. Threats will be dealt with in hours, not days or weeks. AI solutions are being rolled out that can verify code and identify bugs and vulnerabilities, making the digital systems they protect more robust. For example, in April 2017 software firm DarkTrace launched Antigena, which uses machine learning to spot abnormal behaviour on an IT network, shut off communications to that part of the system, and issue an alert. Implementations of AI in cyber security 2 will become increasingly pervasive. According to some estimates, the value of AI in cyber security will reach $18.2 billion by 2023 (Figure 1). Experts welcome the potential of AI to improve system verification and robustness [6]. However, this potential is crippled by two risks: the escalation of responses and the lack of control over snowballing effects that AI-led counter-attacks may have. Thus, it is crucial that experts ensure transparency about problems, limitations, and possible shortcomings of unregulated uses of AI for defence, while also working with policy-makers and end users to design adequate testing and oversight mechanisms for this technology. By the end of this decade, many states plan to deploy AI for national cyber defence [4,5]. AI makes deterrence possible [3] because algorithms can identify the source and neutralise it without having to identify the actor behind it, so that attacks can be punished. Currently, countries hesitate to push back because they are unsure who is responsible, given that campaigns may be waged through third-party computers and often use common software. States need regulations that establish legitimate uses of AI for national defence, shape the process leading to the deployment of AI, such as testing and auditing mechanisms, and define the chain of responsibilities [7]. Cyberspace is a domain of warfare and AI is a new defence capability, so regulations are necessary for state use of AI, much like there exist regulations for the other domains of warfare – air, see, land, and space – and the corresponding military capabilities. What is long overdue are criteria for determining proportional responses; clear thresholds, so called ‘red lines’, for distinguishing legal and illegal cyber-attacks; and appropriate sanctionsfor illegal acts[8]. In each case, unilateral approaches will be ineffective. Rather, it is crucial to define an international doctrine for state action in cyberspace. Alarmingly, international efforts to regulate cyber conflicts have stalled. We call on regional forums, like NATO and EU, to revive efforts and prepare the ground for an UN-led initiative.