Cave and Ó hÉigeartaigh 17

As the applications of AI research become more lucrative, and the impacts of AI on economic, scientific and military applications more transformative, this competition can be expected to intensify. A narrative has begun to emerge, in both policy and public contexts (Simonite 2017, Allen and Husain 2017, Stewart 2017, Allen and Kania 2017, Allen and Chan 2017), framing future trajectories relating to the development of AI in terms typically associated with a race for technological superiority. The China State Council’s 2017 “A Next Generation Artificial Intelligence Development Plan” includes in its aims To seize the major strategic opportunity for the development of AI, to build China’s first-mover advantage in the development of AI The United States National Science and Technology Council’s Research and Development Strategic Plan highlights that the US no longer leads the world in numbers of deep learning-related publications. Strong public statements have been made by political and technology leaders: notable examples include Russia’s President Vladimir Putin stating that “whoever becomes the leader in this sphere will become the ruler of the world” (RT 2017) and OpenAI cofounder Elon Musk tweeting that “competition for AI superiority at national level [is the] most likely cause of WW3” (Musk 2017). These developments reflect a perception that global leadership in AI could confer scientific, infrastructural and economic advantages to frontrunners. Such comparative advantage could also be self-perpetuating, if it helps to recruit the best research talent, or leads to greater wealth and spending power, or to the accumulation of better computing and data resources, or if the application of AI to the process of scientific research itself leads to further breakthroughs. Similarly, although many breakthroughs are likely to be local in application, the perception that some fundamental advances can be applied to a broad range of research challenges further fuels the idea that a winner could take it all. Such notions of ‘winner takes all’ or the importance of technological leadership, foster the framing of AI development as a race. We will refer to this framing as that of the ‘race for technological superiority’ in AI. This kind of race is the primary concern of this paper. But at the same time, it is important to note that there are also concerns about a specifically military AI arms race. Although this is not the primary concern of this paper, there are some noteworthy interrelations with the broader race for technological superiority: most significantly, if AI fulfils its promise as a widelyapplicable and transformational technology, then general superiority in AI will to a significant degree imply also military superiority. Many aspects of AI progress are likely to have dual-use relevance to both civilian and military applications (for example, advances enabling greater autonomy and a wider range of capabilities in autonomous vehicles) not least given the deep connection between AI research and the defence community (Geist 2016). And if systems approach general intelligence or superintelligence, they are likely to confer a very significant strategic advantage that could encompass, but go well beyond, conventional weaponry, with radical implications for security and geopolitics (Allen and Chan 2017). The central concern of this paper is what might happen if the rhetoric noted above around first mover advantage or the pursuit of AI superiority becomes a widespread or even dominant framing of AI development. Our concerns arise both from use of the language of a race for technological superiority (independently of whether such a race is happening) and from such a race becoming a reality. The Dangers of an AI race for Technological Advantage: in Rhetoric and in Reality What is so bad about framing the development of AI in terms of a race for technological advantage? After all, it is widely agreed that AI brings enormous potential benefits across many sectors. One recent report estimated that it could add £232 billion by 2030 to the UK economy alone, with healthcare one of the sectors most enhanced, potentially bringing faster, better service to consumers (PwC 2017). There is a widespread belief that competition and other market forces are central to such innovation. So in as much as a race to develop AI technology means these kinds of benefit come sooner, we have reason to view it positively. But at the same time, the development of such a potentially powerful new technology will need to be steered if it is to be as beneficial as possible while minimising the risks it might pose (Crawford and Calo 2016). In the words of the Future of Life Institute’s open letter on ‘Research Priorities for Robust and Beneficial Artificial Intelligence’, signed by over 8,000 people including many leading figures in AI, work should focus “not only on making AI more capable, but also on maximizing the societal benefit of AI.” (Future of Life Institute, 2017a). The danger of an AI race is that it makes exactly this thoughtful steering towards broadly beneficial outcomes more difficult. Three Sets Of Risks We want to distinguish between three sets of risks: i) The dangers of an AI ‘race for technological advantage’ framing, regardless of whether the race is seriously pursued; ii) The dangers of an AI ‘race for technological advantage’ framing and an actual AI race for technological advantage, regardless of whether the race is won; iii) The dangers of an AI race for technological advantage being won. (i) Risks Posed by a Race Rhetoric Alone It is possible that the trend towards ‘race for technological advantage’ terminology in AI, including suggestions such as Vladimir Putin’s that the winner of such a race “will become the ruler of the world,” could pose risks even if the race is not pursued in earnest, let alone won. We perceive two main risks here: (i.a) The kind of thoughtful consideration of how to achieve broadly beneficial AI, as mentioned above, will require subtle, inclusive, multi-stakeholder deliberation over a prolonged period. The rhetoric of the race for technological advantage, with its implicit or explicit threat that dire consequences will follow if some other group wins that race, is not likely to be conducive to such deliberation. Indeed, rhetoric around technological superiority (such as the ‘arms race’ rhetoric used in the Cold War in the US), played into what has been called a politics of fear, or a politics of insecurity -- that is, a political climate that discourages debate in favour of unquestioning support for a prescribed agenda (Griffith 1987). In The Politics of Insecurity, Jef Huysmans argues that use of the language of security (by which he means militarised language, which would include ‘arms race’ and related rhetoric) “is a particular technique of framing policy questions in logics of survival with a capacity to mobilize politics of fear in which social relations are structured on the basis of distrust” (Huysmans 2006). (i.b) Second, if the rhetoric of a competitive, ‘winner takes all’ AI race is used in the absence of an actual race, it could contribute to sparking such a race. (ii) Risks Posed by a Race Emerging If the rhetoric of a race for technological advantage became an actual race to develop sophisticated AI, the risks increase further: (ii.a) First, there is the risk that racing to achieve powerful AI would not be conducive to taking the proper safety precautions that such technology will require (Armstrong, Bostrom, and Shulman 2016). We mentioned above the need for broad consultation about the role AI should play in the life of a community. This might help address important considerations such as avoiding biased systems, or maximising fairness. But in addition to these goals, serious attention must also be given to ensuring humans do not lose control of systems. Such considerations become particularly important if AI approaches general intelligence or superintelligence (Bostrom 2014), but also long before, particularly when AI systems are performing critical functions. The risk is that as the perceived benefit to winning the race increases, so correspondingly does the incentive to cut corners on these safety considerations. (ii.b) Second, a ‘race for technological advantage’ could increase the risk of competition in AI causing real conflict (overt or covert). Huysmans argues that militarised language such as this has “a specific capacity for fabricating and sustaining antagonistic relations between groups. In the case of the race for technological advantage, it encourages us to see competitors as threats or even enemies. The belief that a country intends in earnest to win an AI race, and that this would result in technological dominance, could, for example, prompt other countries to use aggression to prevent this (akin to the cyberattacks made against Iranian nuclear facilities attributed to the US and Israel) (Nakashima 2012), or motivate the targeting of key personnel (precedents -- though specific to their historical context -- might include Operation Paperclip, during which over 1,600 German scientists and engineers who had worked on military technology were taken to the US (Jacobsen 2014), or the apparently ongoing operations to encourage the defection of nuclear scientists between nations) (Golden 2017). Such scenarios would also increase the risk that a general race for technological superiority became increasingly a military AI arms race. (iii) Risks Posed by Race Victory The third category of risks of an AI race for technological superiority are those that would arise if a race were won. We will not explore these in detail here -- and the forms they take will anyway depend on the precise nature of the technology in question. But as an example, these risks include the concentration of power in the hands of whatever group possesses this transformative technology. If we survey the current international landscape, and consider the number of countries demonstrably willing to use force against others, as well as the speed with which political direction within a country can change, and the persistence of non-state actors such as terrorist groups, we might conclude that the number of groups we would not trust to responsibly manage an overwhelming technological advantage exceeds the number we would.