Chinese lead encourages the worst misuses of AI – extinction
James Johnson 19 {Dr. Johnson holds a Ph.D. in Politics & International Relations from the University of Leicester, an M.A. in Asia-Pacific Studies from the University of Leeds. 4-24-2019. “Artificial intelligence & future warfare: implications for international security.” https://www-tandfonline-com.proxy.lib.umich.edu/doi/full/10.1080/14751798.2019.1600800}//JM
China’s initial approach to AI has been heavily influenced by its assessment of U.S. military initiatives; in particular, those associated with the DoD’s Third Offset Strategy, and more recently, “Project Maven” (e.g. human-machine collaboration; convolutional neural networks; big-data analytics; machine-learning; human-assisted operations; combat-teaming; and autonomous weapons).108 108. China’s long-standing approach to military innovation has been based on a “leap-frogging” strategy; designed to encourage civil-military collaboration in the development of dual-use asymmetric capabilities. View all notes As China’s approach to AI matures, however, it will more likely align closer with the People’s Liberation Army’s (PLA’s) unique organisational, command and control, and strategic cultural traditions.109 109. Johnson, The US-China Military and Defense Relationship during the Obama Presidency, chap. 4. View all notes Beijing, like the U.S., has yet to formally articulate a coherent strategic framework, operational concepts, or the establishment of institutions and mechanisms to support the use of AI for warfighting.110 110. For a recent study on Chinese approaches to weapon system-related operational concepts see, Jeffrey Engstrom, Systems confrontation and system destruction warfare (Santa Monica: RAND Corporation, 2018). View all notes That said, the intensity of discussion and research within the PLA surrounding military-use AI is indicative of the high-level importance attached to this ubiquitous dual-use technology.111 111. ‘National People’s Congress Representative Liu Guozhi: Artificial Intelligence Will Accelerate the Process of Military Transformation’, PLA Daily, March 7, 2017, http://jz.chinamil.com.cn/zhuanti/content/2017-03/07/content_7517615.htm/. View all notes As China and the U.S. internalise these emerging technological trends, it is likely that each side will conceptualise them very differently. Scholarship on military innovation has demonstrated that – with the possible exception of nuclear weapons – technological innovation alone rarely causes the military balance to shift; rather how militaries employ a technology usually proves critical.112 112. See, Barry R. Posen, The sources of military doctrine: France, Britain, and Germany between the world wars (Ithaca: Cornell Studies in Security Affairs, 1986). View all notes A major cause for concern is that if the many national, cultural, and normative differences that separate Sino-American approaches to military innovation are reflected in the software used to teach AI programmes, the resultant prejudices and preferences might become baked into the weapon systems they support. 113 113. For example, Microsoft’s racist “Chatbot Tay” is the most infamous example of this kind of prejudice displayed based on the data and parameters used by developers. View all notes As a corollary, even if AI systems are designed to produce bias-free analysis, human bias inherent in data sampling, sensor types, and other uncontrollable factors, might nonetheless result in subjective decision-making.114 114. However, if future AI is able to collect and categorise its own data via sensors, then the susceptibility of machines to human biases will likely decrease. For a history of AI and the military see, Ayoub and Payne, ‘Strategy in the Age of Artificial Intelligence’, 793–819. View all notes Under crisis and conflict conditions, these kinds of cognitive biases might exacerbate underlying U.S.-China mutual mistrust, suspicion, and misperceptions. In the race to innovate in AI, uncertainties surrounding U.S and China progress (and setbacks) will have profound and potentially destabilising implications for the strategic balance.115 115. China and the United have developed the capability to leverage AI to achieve asymmetric combat advantages, but its employment will also introduce certain vulnerabilities. Moreover, there will likely be continued obstacles to the effective sharing, acquisition, and fielding of AI systems for military applications. View all notes For now, at least, the U.S. retains the upper-hand in AI innovation,116 116. The U.S. leads China in the number of AI patent applications, the number of AI-related organisations, the amount of funding provided, but China is quickly closing this gap. View all notes but in this emerging innovation arms-race, China is no longer the inferior party. Instead, China is fast becoming a true peer-competitor in AI and is expected to soon overtake the U.S. in this emerging strategic domain.117 117. International Institute for Strategic Studies (IISS), The Military Balance, 2018 (London: IISS, 2018), 10–3. View all notes By its own estimates, Beijing has set 2020 as a target to achieve “major breakthroughs in a series of landmark AI products”, and to establish an “international competitive advantage” in the development of dual-use technologies and applications – especially those which target the United States.118 118. From 2014, China has surpassed the United States in the output of published research papers on deep learning - by circa 20 per cent in 2016 alone. While increases in the quantity of AI-related publications do not necessarily correspond to advances in quality, this trajectory nonetheless, clearly demonstrates that China is resolutely committed to its AI development agenda. View all notes To be sure, China’s innovation ambitions could be expedited by a fundamental mismatch (even dissonance) analysts have identified between the rapid pace of commercial innovation and academic research into AI and the lagging timescales and assumptions that underpin the Pentagon’s existing procurement processes and practices.119 119. Andrew Ilachinski, AI, Robots, and Swarms - Issues, Questions, and Recommended Studies (Washington: CNA Analysis and Solutions, 2017), xiv. View all notes Chinese centralised planning, socialist market economy, and in particular, a vast pool of data-sets, could offer Beijing significant scope to leverage China’s market forces and human capital to realise its “civil-military fusion” objective in AI.120 120. Beijing’s approach to AI is, however, far from perfect. Chinese state-led resource management characterised as inefficient and intrinsically corrupt (with government-favoured research institutions receiving a disproportionate share of state-funding) might cause the government to misallocate resources, over-invest in non-productive and poorly conceptualised AI projects. View all notes While vast data is clearly an advantage, however, it remains an open question whether China’s national strategic planning, and socialist market economy will prove advantageous in the development of AI. According to a recent report, China is on track to possess twenty per cent of the world’s entire data by 2020 – and thirty per cent by 2030.121 121. In contrast, between 2012–2017 U.S. DoD expenditure on AI-related contracts was relatively flat. Govini, ‘Department of Defense Artificial Intelligence, Big Data, and Cloud Taxonomy’, December 3, 2017, 9, http://www.govini/home/insights/. View all notes The head of the U.S. DoD’s Strategic Capabilities Office, William Roper, highlighted the pivotal role the accumulation of, and competition for, information for machine learning will play in future warfare. Roper stated: “It’s wealth and fuel. Your data keeps working for you. You stockpile the most data that you can and train that to teach and train autonomous systems”.122 122. Patrick Tucker, ‘The Next Big War Will Turn on AI, Says US Secret-Weapons Czar’, Defense One, 28, March 2017, https://www.defenseone.com/technology/2017/03/next-big-war-will-turn-ai-says-pentagons-secret-weapons-czar/136537/. View all notes In contrast to the nuclear arms race that defined the Cold War-era, states competing in the AI arms race will be less concerned with sustaining the qualitative and quantitative lead in warheads, but instead will be more concerned with maintaining information superiority – to feed machine-learning algorithms.123 123. Sharikov, ‘Artificial intelligence, cyberattack, and nuclear weapons - A dangerous combination’, 370. View all notes Chinese President Xi Jinping recently stated that AI, “big data”, cloud storage, cyberspace, and quantum communications were amongst the “liveliest and most promising areas for civil-military fusion”, and towards this end, he pledged additional state support and resources.124 124. For example, in collaboration with Baidu, Beijing established a “National Engineering Laboratory of Deep Learning Technology” initiative. Robin Li, ‘China brain project seeks military funding as Baidu makes artificial intelligence plans’, South China Morning Post, 3 March, 2015, https://www.scmp.com/lifestyle/article/1728422/china-brain-project-seeks-military-funding-baidu-makes-artificial. View all notes In contrast, the increasingly strained relationship between the Trump administration and Silicon Valley will likely pose additional challenges to this critical partnership in the development of AI technologies for the U.S. military.125 125. For example, when Google acquired DeepMind it specifically prohibited the use of its research for military purposes. Loren DeJonge Schulman, Alexandra Sander, and Madeline Christian, ‘The Rocky Relationship Between Washington & Silicon Valley: Clearing the Path to Improved Collaboration’, (Washington: CNAS, July 2015). View all notes Following a recent high-profile backlash from employees at Google, the company recently announced that it would discontinue its work with the Pentagon on Project Maven.126 126. Jeremy White, ‘Google Pledges not to work on weapons after Project Maven backlash’, The Independent, 7 June, 2018, https://www.independent.co.uk/life-style/gadgets-and-tech/news/google-ai-weapons-military-project-maven-sundar-pichai-blog-post-a8388731.html. View all notes As a first mover AI-power, therefore, China will likely chart a course to be at the vanguard in the development of technical standards, mechanisms, and governance of AI that will likely strengthen the competitiveness and quality of China’s military capabilities.127 127. Given the lack of empirical open-sources that relates to Chinese view on military applications of AI, this paper highlights some of the key observable trends, and proffers areas for future research that relates to these issues. View all notes China’s early approach to AI suggests a wide-reaching conceptualisation that the PLA will synthesise into its entire force structure; to support future “intelligentised” operations, and seise the “commanding heights” of future strategic competition.128 128. ‘National People’s Congress Representative Liu Guozhi: Artificial Intelligence Will Accelerate the Process of Military Transformation’, PLA Daily, March, http://jz.chinamil.com.cn/zhuanti/content/2017-03/07/content_7517615.htm/. View all notes Specifically, Chinese researchers have focused on AI applications for war-gaming, training, command and control, intelligence analysis, and augmenting autonomous weapons systems.129 129. Shou Xiaosong, ed., The Science of Military Strategy, 3rd ed. (Beijing: Military Science Press, 2013). View all notes President Xi’s “One Belt One Road”, and the virtual dimension the “digital Silk Road”, are high-level efforts designed to ensure that the mechanisms, co-ordination, and support for this agenda will become increasingly normalised.130 130. China’s recent five-year plan reportedly committed over USD$100 billion to AI. Moreover, as China moves forward with its One Belt One Road related projects that extend to potentially more than eighty countries AI would become an integral part of these international infrastructure projects. Wenyuan Wu, ‘China’s Digital Silk Road: Pitfalls Among High Hopes’, The Diplomat, 3 November, 2017, https://thediplomat.com/2017/11/chinas-digital-silk-road-pitfalls-among-high-hopes/. View all notes Moreover, in 2017 Xi explicitly called for the acceleration of the military “intelligentisation” agenda, to better prepare China for future warfare against a near-peer adversary like the United States.131 131. ‘Xi Jinping’s Report at the 19th Chinese Communist Party National Congress’, Xinhua, October 27, 2017, http://www.china.com.cn/19da/2017-10/27/content_41805113_3.htm. View all notes China’s pursuit of AI (especially dual-use capabilities) will fuel the perception (accurate or otherwise) in Washington that Beijing is intent on exploiting this strategically critical technology to fulfil its broader revisionist goals. Despite a brief pause in the development of the U.S.’s AI strategic roadmap, the White House recently announced the creation of a new committee of AI experts to advise it on policy choices.132 132. Aaron Boyd, ‘White House Announces Select Committee of Federal AI Experts’, Nextgov, May 10, 2018, https://www.nextgov.com/emerging-tech/2018/05/white-house-announces-select-committee-federal-ai-experts/148123/. View all notes In 2017, following the recommendation of the Committee on Foreign Investment in the U.S., President Trump blocked a Chinese firm from acquiring Lattice Semiconductor; a company that manufactures chips critical in the operation of AI applications.133 133. Ana Swanson, ‘Trump Blocks China-Backed Bid to Buy U.S. Chip Maker’, The New York Times, September 13, 2017, https://www.nytimes.com/2017/09/13/business/trump-lattice-semiconductor-china.html. View all notes This action typifies a broader concern that synergies created by China’s civil-military fusion strategy could allow the technology, expertise, and intellectual property shared between American and Chinese commercial entities to be transferred to the PLA.134 134. Bartholomew and Shea, U.S.-China Economic and Security Review Commission - 2017 Annual Report, 507. View all notes Though Chinese strategic writings have emphasised the importance of human-machine collaboration and teaming (or keeping humans “in the loop”),135 135. “Keeping humans in the loop” refers to maintaining human control of autonomous weapons; both in the design of the rules that govern these systems, and the execution of those rules when firing. That said, human decision-making and automation are not necessarily mutually exclusive. For example, the human-machine teaming cognitive design envisaged by the Pentagon, in theory at least, could leverage the predictability, reliability, and speed of full-automation while retaining the robustness and flexibility of human intelligence. View all notes the PLA’s historical resistance to command and control decentralisation, and general mistrust of human personnel could prompt military leaders to gravitate more quickly towards full-battlefield autonomy.136 136. For a recent comprehensive examination of the PLA’s shortcomings see, Michael S. Chase, Jeffrey Engstorm, Tai Ming Cheung, Kirsten A. Gunness, Scott W. Harold, Susan Puska, and Samuel K. Berkowitz, China's incomplete military transformation- assessing the weaknesses of the people's liberation army (PLA) (Santa Monica: RAND Corporation, 2015). View all notes The opposite conclusion could also be drawn, however: if Chinese commanders were unwilling to give up centralised control to junior officers, why would they give such control to machines? Recent reports indicate China’s navy is contemplating fitting its nuclear-powered submarines (and possibly nuclear-armed ones) with a so-called “AI-augmented brainpower”.137 137. Stephen Chen, ‘China’s plan to use artificial intelligence to boost the thinking skills of nuclear submarine commanders’, South China Morning Post, 4 February, 2018, https://www.scmp.com/news/china/society/article/2131127/chinas-plan-use-artificial-intelligence-boost-thinking-skills. View all notes This capacity could, in theory, synthesise and interpret large quantities of data generated by sonar signals and sound pulses, to detect submerged objects, and support a broad range of maritime operations. To be sure, the kinds of operations and the level of autonomy afforded to AI-augmented systems to support China’s strategic underwater forces will have profound implications for future crisis and conflict in the increasingly contested undersea domain. In extremis, if military command and control systems came under attack (possibly from AI-augmented cyber-weapons), military commanders may decide to pre-delegate decision-making to machine-learning systems. Russia, for example, operates a so-called “dead hand” designed to automatically launch its nuclear missiles at hyper-speed, if its pressure sensors were to detect an imminent nuclear attack.138 138. Fontaine and Miller, A new era in U.S.-Russian strategic stability, 26. View all notes The evidence suggests that China (and Russia) has relatively few moral, legal or ethical qualms in deploying lethal autonomous weapons.139 139. To date, there have been few publications on the legal and ethical implications for military-use AI, which have dominated the discourse in the West. Bendett, ‘Get Ready, NATO.’ View all notes Moreover, and in contrast to the U.S., discussion on the potential limitations and risks associated with AI, autonomy and cyber-warfare appears largely absent from Chinese open-sources.140 140. For example, Johnson, The US-China Military and Defense Relationship during the Obama Presidency, chap. 4. View all notes Reports suggest that China has already begun to incorporate AI into its next-generation conventional missiles and missile-defense intelligence gathering systems, to enhance their precision and lethality.141 141. Kania, Battlefield singularity. View all notes By contrast, the U.S. will likely be much more constrained in the development of these technologies. Resistance within the U.S. military to incorporate AI stems in large part from the prevailing liberal-democratic norms governing the use of military force, and the growing concerns surrounding the many “black box” aspects of AI-machine learning, and in particular, to avoid the so-called “Terminator Conundrum” – the implications of weapons that could operate independently and beyond the control of their developers.142 142. Colin Clark, ‘“The Terminator Conundrum:” VCJCS Selva On Thinking Weapons’, Breaking Defense, January 21 2016, https://breakingdefense.com/2016/01/the-terminator-conundrum-vcjcs-selva-on-thinking-weapons/. View all notes Chinese analysts, by overlooking the potential shortcomings, uncertainties, and vulnerabilities associated with AI, and overstating (even overdramatising) the utility of AI and autonomy (or taking humans “out of the loop”), could under crisis and conflict conditions complicate escalation management,143 143. For example, whilst much has been written by Chinese analysts on the Pentagon’s Third Offset Strategy programmes (including AI) there has been very little discussion on the potential limitations of these advanced systems - including those associated with reducing human control. View all notes and worsen strategic stability in future warfare.144 144. Geist and Lohn, How might artificial intelligence affect the risk of nuclear war? (Santa Monica: RAND Corporation, 2018), 5. View all notes That said, given the aggressive pursuit of military-use AI by its strategic rivals, America’s current commitment to having humans in charge might waver.145 145. It remains unclear, however, what operational contexts and applications, and to what degree China and Russian might pursue fully autonomous weapon systems. View all notes Moreover, international law remains unclear and indeterminate on lethal autonomy, and in its absence, militaries (including the U.S.) will continue to develop weapon systems with varying degrees of autonomy.146 146. Kelsey Atherton, ‘3 big takeaways from the Navy’s new robot road map’, C4ISRnet, 30 May, 2018, https://www.c4isrnet.com/unmanned/2018/05/30/three-big-takeaways-from-the-navys-new-robot-roadmap/. View all notes Ultimately, militaries will need to consider the trade-off between the risks associated with autonomous weapons, with the possibility of affording an adversary using fully autonomous weapons the asymmetric upper hand. At this early stage, it is impossible to know for certain when, whether, and under what circumstances greater degrees of autonomy in human-machine collaboration will provide a distinct strategic battlefield advantage.