Garcia 23

Instability and insecurity The major military powers have been redefining their national security in terms of their pursuit of superiority in the realm of AI. Many countries have already announced their national plans to do just this. The Chinese have termed the search for AI-assisted weapon systems as “intelligentized warfare,” which represents a military revolution characterized by ubiquitous data-rich networks of autonomous and sensor-based intelligent systems.11 The advent of the fourth industrial revolution creates incentives for states to include not only AI but also quantum, nanotechnology, biotechnology, and other forms of technology into their military spheres.12 The fourth industrial revolution is qualitatively differently from the preceding ones by rapidly fusing the biological, digital, and physical domains of life on Earth. The technologies that fall under the umbrella term “AI” feature prominently. Indeed, the Chinese approach to “intelligentized warfare” epitomizes the third revolution in warfare. The confluence of the fourth industrial revolution and the third revolution in warfare may create some opportunities, but also harbingers risks. The militarization and the weaponization of AI are both underway and this phenomenon will impact on international peace and security in appreciable ways. It is already creating a renewed sense of rivalry and competition that is manifesting in the quest to develop more weapon systems. The allocation of funds devoted to AI systems has also been on the rise (Chapter 1).13 Will this march toward including AI lead to a redistribution of power and a sense that world security and stability are threatened? Along with other scholars, I contend that the impact of AI will be disruptive of the prevailing, yet volatile, balance of power.14 The need to preserve strategic stability is where there may be some common ground: self-interested states always work toward maintaining their own national security. In principle, all countries want to maintain international stability to carry on commerce, trade, and other human activities. It is worth noting that “strategic stability” is a concept related to nuclear stability, the centrality of the notion of deterrence, and the absence of incentives to use nuclear weapons first. The concept then came to encompass more widely the reliance on conventional weapons, cyber security, and emerging technologies. However, it continues to be based on the idea that the international system is changeable and precarious.15 This perpetuates the view that more arms mean greater security. A more appropriate view would be to regard strategic stability as the consequence of a global security environment wherein states can maintain peaceful and harmonious international relations.16 The major military powers see AI as a critical magnifier of their present and future nuclear capabilities. Most of the countries seeking to militarize AI also possess nuclear weapons. According to the authoritative assessment by the Stockholm International Peace Research Institute (SIPRI), machine learning algorithms are already unpredictable and lack transparency (i.e., it is hard to understand how the machine arrives at the outcome). Therefore, command and control of nuclear systems are too critical to international security to be left to algorithms. This signals an agreement not to integrate machine learning and autonomy into the command-and-control systems of nuclear weapons.17 Nonetheless, the nuclear states do see AI as a vital amplifier of their future military capabilities.18 The risks are not trivial, though. The use of AI may heighten the risk of inadvertent use and, as a result, an escalation that could be magnified because of the loss of human control. The unpredictability and unreliability of AI could make for a precarious international system.19 The costs and benefits of the use of AI must be considered against the background of the broader geopolitical landscape, and how they may cause instability. Altmann and Sauer warn that the perils of operating machine-learning weapons reside in the inability to accurately foresee all outcomes in advance.20 This is especially the case when different countries’ algorithms interact with each other during operations. This lack of capacity to predict outcomes hampers the ability of humans to act as overseers in case things go wrong. Machine learning—succinctly defined by Sauer as the use of algorithms and advanced statistical models to improve task performance—is still limited to an extremely narrow set of tasks. Machine learning is “greedy (i.e., hungry for immense amounts of data), brittle (i.e., failing when confronted with a task that differs slightly from what it was trained for), and opaque (i.e., generating unexplainable outputs, essentially rendering it a black box that is impossible to debug).”21 It is worth bearing in mind the two dimensions of instability: the first is arms proliferation and a horizontal and vertical arms race.22 This refers, firstly, to proliferation beyond militarized countries and, secondly, to destabilizing accumulations by key military powers that would exacerbate the security dilemma. Autonomous systems are easier to build than nuclear weapons, which require extensive apparatus such as centrifuges, high-speed fuses, and controlled use of rare raw materials. Autonomous systems are likely to become available to a wider range of actors, including nonstate actors, because their implementation is more about the software. The implication of this is that a potentially destabilizing military race is already taking place. The centrality of software in autonomous systems makes them less reliant on traditional, quantitative, weapons-counting arms control. The software can be more readily duplicated and interfered with by manipulating computer network operations. On the battlefield, characterized by a dearth of data, this means that unpredictability and vulnerability compound uncertainty. The second dimension of instability is escalation due to accidents or lack of communication.23 Three factors are relevant here: casualty avoidance, cost reduction, and the implications of weapon swarming (the use of small or miniaturized systems operating together); this explains why it is likely that countries would more often resort to autonomous systems in surprise attacks that could lead to crises. Casualty avoidance is prompted by the attempt to keep soldiers out of harm’s way and reduce political risks, especially in democracies. Autonomous systems may lead to cost reductions as 3D printing becomes increasingly common. Autonomy can bring military advantages but creates vulnerabilities that create risks because it removes the human as the overseer. The costs of the deployment of autonomous weapon systems far outweigh the benefits. Global security will be harmed as more actors, many not accountable to legal frameworks under international law, start to play a more prominent role by using these systems. The interaction of different-origin algorithms increases the likelihood of the precipitous escalation of a crisis into violent conflict.24 For all these reasons, AI may not reach the levels of reliability and trustworthiness required by many military applications where life is at stake. What is certain is that the pursuit of the weaponization of AI is indeed leading to a greater risk of military races.25 Furthermore, the premature application of AI in the military may reduce and not enhance compliance with IHL. In sum, new global governance is the guardrail needed to foster prudence to avert a reckless AI military race toward the promised yet unfulfilled role of AI technologies.