Free beta access. 50 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Drone proliferation is rapid and escalates without checks

Altmann & Sauer 17 – (Jürgen Altmann, lecturer in experimental physics at Technical University of Dortmund, working on the prospective assessment of new military technologies and the analysis of preventive arms-control measures, Frank Sauer, senior research fellow and lecturer in international relations at Bundeswehr University in Munich, working on international security and arms control, “Autonomous Weapon Systems and Strategic Stability,” Global Politics and Strategy, Volume 59, 2017, Iss. 5, https://www.tandfonline.com/doi/abs/10.1080/00396338.2017.1375263?journalCode=tsur20)

Proliferation and arms-race instability As early as 2007, the US Department of Defense wrote in its Unmanned Systems Roadmap that for processor technology ‘the ultimate goal is to replace the operators with a mechanical facsimile [of] equal or superior thinking speed, memory capacity, and responses gained from training and experience’. The document also stated that the ‘primary technical challenges for weapon release from unmanned systems include the ability to reliably target the right objective’.17 The goal of weapon autonomy pervades all subsequent road maps.18 Autonomous weapon-system functions have since been tested on land, under water, on the sea and, most notably, in the air. In fact, current trends with respect to unmanned combat aerial vehicles (UCAVs or ‘combat drones’) provide indicators for what to expect with regard to AWS. Unlike today’s high-profile UCAVs, such as the Reaper, which are propeller driven, slow, carry comparably small payloads and have few to no capabilities for operating in contested airspace, future systems will be less dependent on human control, faster, stealthy and capable of delivering bigger payloads. The X-47B, for instance, has demonstrated autonomous take-off from and landing on an aircraft-carrier deck, as well as autonomous aerial refuelling. This technology demonstrator was developed by the US Navy’s Unmanned Carrier-Launched Airborne Surveillance and Strike programme (UCLASS). Similarly, the British Taranis UCAV was described by the UK Ministry of Defence as ‘fully autonomous’ and able to ‘defend itself against manned and other unmanned enemy aircraft’ with ‘almost no need for operator input’.19 However, the ministry also stated that ‘the operation of weapons systems will always be under human control’.20 While AWS test beds such as Taranis and the X-47B rely on familiar designs, in this case the airframes of a fast, stealthy, next-generation drone with substantial payload capabilities, future systems will display an autonomous swarming capability, and thus AWS will also come in much smaller sizes. In October 2016, for instance, the US Department of Defense demonstrated a swarm of 103 Perdix micro drones capable of ‘advanced swarm behaviors such as collective decision-making, adaptive formation flying, and self-healing’.21 In the future, such micro drones are to be 3D printed in large batches and deployed from (manned) flying systems. This dispensing method has already been successfully tested at Mach 0.6 speed by two F/A-18 Super Hornets releasing a Perdix drone swarm. The US Navy’s LOCUST programme is also seeking to develop swarming, disposable unmanned aerial vehicles (UAVs).22 The overall goal for this new ecosystem of flying assets is to replace not just the old generation of drones but also manned aircraft, thus continuing the trend towards keeping human pilots out of harm’s way and providing superior unmanned air-to-ground and air-to-air capabilities across the board.23 In air-to-air combat, the big, fast autonomous drones currently envisioned will be able to fly high-g manoeuvres no human pilot would be able to endure. More importantly, they would ensure much shorter reaction times. On-board sensors combined with artificial ‘intelligence’ – either located onboard or distributed in the swarm and based on decision-making algorithms endowed with the authority to initiate an attack without awaiting human input – are to make these weapons autonomous and hence provide a decisive edge over remotely controlled and human-piloted adversary systems alike. While the development of AWS is currently most advanced in the air and under water – that is, in less cluttered environments – the example of autonomous (swarms of) UCAVs demonstrates the generally valid proposition that for future unmanned systems, operational speed will reign supreme, regardless of the domain. In that sense, technological developments in AI and robotics, as well as current expectations regarding future armed conflict (and the need for speed), jointly point towards AWS. In fact, US deputy secretary of defense Bob Work stated in March 2016 that even the final delegation of lethal authority to autonomous systems will inexorably happen as a result of this race for speed.24 According to Work, the United States ‘will not delegate lethal authority for a machine to make a decision … The only time we’ll delegate authority is in things that go faster than human reaction time, like cyber or electronic warfare.’ Yet, he conceded that such self-restraint may be unsustainable if an authoritarian rival acts differently. ‘We might be going up against a competitor who is more willing to delegate authority to machines than we are and, as that competition unfolds, we’ll have to make decisions on how we can best compete’, Work said. ‘It’s not something that we have fully figured out, but we spend a lot of time thinking about it.’25 To further deepen our understanding of AWS, it is useful to take a step back and underline that they need not necessarily take the shape of a specific weapon system akin to, for instance, a drone or a missile. AWS also do not require a specific military-technology development path, the way nuclear weapons do, for example. As AI, autonomous systems and robot technologies mature and begin to pervade the civilian sphere, militaries will increasingly be able to make use of them for their own purposes, as the development of information and communication technology suggests. Naturally, any military adaptation of a dual-use technology will need to fulfil specific military requirements that do not exist in a civilian environment, or are less relevant for mass markets. Nevertheless, AWS development will profit from the implementation or mirroring of a variety of civilian technologies (or derivatives thereof) and their adoption for military purposes, technologies which are currently either already available or on the cusp of becoming ready for series production in the private sector. This trend is already observable in the case of armed drones. Light detection and ranging (LIDAR) systems are another example. These are the optical sensors used by the automotive industry to give self-driving cars a 360-degree picture of their surroundings. LIDAR prices have recently dropped from five figures to a few hundred dollars. The units have also become more rugged and much smaller.26 Given that these components, which are necessary for endowing mobile systems with autonomy, are now cheaply and readily available off the shelf, there is every reason to expect the military to adapt, and, if required, adjust and refine, them for their own purposes.27 It is clear that the research and development for AWS-relevant technology is well under way and distributed across countless university laboratories and, especially, commercial enterprises that are making use of economies of scale and the forces of the free market to spur competition, lower prices and shorten innovation cycles. This renders the military research and development effort in the case of AWS different from those of past high-tech conventional weapon systems (the F-35 comes to mind), let alone nuclear weapons. So while the impact of AWS might be revolutionary in terms of their implications for warfare, their development within the context of the military is best described as evolutionary: the military is merely continuing and, with outside help and technology lifted from the private sector, accelerating an already existing trend to replace labour with capital and automate dull, dirty and dangerous military tasks.28 For example, former secretary of defense Ashton Carter sought closer ties with Silicon Valley to hasten the incorporation of technological innovations into the US military after the US officially declared AI and robotics cornerstones of its new ‘third offset’ strategy to counter rising powers.29 Thus, AWS are easy to obtain compared with other paradigm-shifting weapons, such as nuclear weapons, which even now require the Herculean effort of a state-run, focused politico-military effort to produce. AWS do not require ores, centrifuges, high-speed fuses or other comparably ‘exotic’ components to be assembled and tested in a clandestine manner. Consequently, while nuclear technologies can be – and are – proliferation controlled, AWS are much harder to regulate. With comparatively fewer choke points that might be targeted by non-proliferation policies, AWS are potentially available to a wide range of state and non-state actors, not just those nation-states that are willing and able to muster the considerable resources needed for the robotic equivalent of the Manhattan Project. 30 This carries significant implications for arms control. There will of course be differences in quality. Sophisticated AWS will have to meet the same or similar military standards that current weapon systems, such as main battle tanks or combat aircraft, do. Moreover, technologically leading nations such as the US and Israel are carrying out research to produce autonomous systems that comply with international humanitarian law. Less scrupulous actors, however, will find AWS development much easier. Comparably crude AWS which do not live up to the standards of a professional military in terms of reliability, compliance with international humanitarian law or the ability to go head-to-head with systems of a near-peer competitor could, in fact, be put together with technology available today by second- or third-tier state actors, and perhaps even non-state actors. Converting a remotely controlled combat drone to autonomously fire a weapon in response to a simple pattern-recognising algorithm is already doable. Even the technological edge displayed by sophisticated AWS is unlikely to be maintained over the longer term. While sensor and weapon packages to a large degree determine the overall capabilities of a system, implementing autonomy ultimately comes down to software, which is effortlessly copied and uniquely vulnerable to being stolen via computernetwork operations. Thus, while the development of AWS clearly presents a challenge to less technologically advanced actors, obtaining AWS with some degree of military capability is a feasible goal for any country already developing, for example, remotely controlled armed UAVs – the number of which rose from two to ten between 2001 and 2016.31 Admittedly, the US and Israel are still in the lead with regard to developing unmanned systems and implementing autonomous-weapon functionality – China only recently test-fired a guided missile from a drone via satellite link for the first time.32 But considering that drone programmes can draw from the vibrant global market for unmanned aerial vehicles of all shapes and sizes, the hurdles regarding AWS are much lower than those of other potentially game-changing weapons of the past. Proliferation of AWS could of course also occur via exports, including to the grey and black markets. In this way, autonomous systems could fall not only into the hands of technologically inferior state actors, but also those of non-state actors, including extremist groups. Hamas, Hezbollah and the Islamic State have already deployed and used armed drones. As sensors and electronics are increasingly miniaturised, small and easily transportable systems could be made autonomous with respect to navigation, target recognition, precision and unusual modes of attack.33 Terrorist groups could also gain access to comparably sophisticated systems that they could never develop on their own. Again, autonomy in this context does not necessarily require military-grade precision – a quick and dirty approach would suffice for these actors. In fact, it stands to reason that terrorist groups would use autonomous killing capabilities indiscriminately in addition to using them, if possible, in a precise fashion for targeted assassinations. It is still unclear how the development of unmanned systems on the one hand and specific countermeasures on the other will play out. Traditional aircraft-sized drones such as the X-47B or Taranis, to stick with these examples, are obviously susceptible to existing anti-aircraft systems. As for smaller-sized systems, various tools, from microwaves to lasers to rifle-sized radio jammers for disrupting the control link, are currently being developed as countermeasures. Simpler, less exotic methods such as nets, fences or even trained hunting birds might also prove effective for remotely controlled and autonomous systems alike. It is clear, however, that saturation attacks have been identified as a key future capability for defeating a wide range of existing and upcoming defensive systems – both human-operated and automatic.34 The latter are a particular focus of research into swarming as a potential solution.35 And military systems operating at very high speeds and in great numbers or swarms are bound to generate new instabilities, to which we will turn in our next section. To first sum up our argument so far, there are obvious dual-use problems and an unusually high risk of proliferation when it comes to AWS. Should one of the technologically leading nation-states go forward with the deployment of AWS, it would be comparably easy – and thus very likely – that others would follow suit.36 In that sense, the development of AWS could well trigger a destabilising arms race. Crisis instability and escalation Increasing operational speeds mean that human involvement in AWS would be limited to, at best, general oversight and decision-making in instances where communication delays of up to a few seconds – and thinking and deliberation times of a few minutes – could be deemed acceptable, meaning they would not result in defeat or the loss of systems. Many situations would not allow for the luxury of human pondering, however. In such cases, the actions and reactions of individual AWS, as well as AWS swarms, would have to be controlled autonomously by algorithms – in other words determined only by programming software in advance and possibly through the adaptation and learning of the systems themselves. After all, as Paul Scharre put it, ‘winning in swarm combat may depend upon having the best algorithms to enable better coordination and faster reaction times, rather than simply the best platforms’.37 One such swarm-combat situation could be a severe political crisis in which adversaries believe that war could break out. With swarms deployed in close proximity to each other, control software would have to react to signs of an attack within a split-second time frame – by evading or, possibly, counter-attacking in a use-them-or-lose-them situation. Even false indications of an attack – sun glint interpreted as a rocket flame, sudden and unexpected moves of the adversary, or a simple malfunction – could trigger escalation. The nature of military conflict is such that these kinds of interactions could not be tested or trained for beforehand. In addition, it is, technically speaking, impossible to fathom all possible outcomes in advance. Clearly, the interaction of swarms, if fully autonomous, would be unpredictable, and could potentially result in an escalation from crisis to war, or, within armed conflict, to higher levels of violence. This is not a theoretical proposition deduced solely from systems theory and the argument of unavoidable ‘normal accidents’.38 On the contrary, comparable runaway interactions between algorithms are already happening in the civilian sphere on a regular basis. In April 2011, the price of an out-of-print biology textbook rose within weeks to $23.7 million on the Amazon marketplace due to the price-setting algorithms of two vendors interacting with each other.39 Eventually one of the vendors intervened; no damage was done because nobody purchased the book at this absurd price. Greater havoc was caused in the New York Stock Exchange ‘flash crash’ of 6 May 2010 in which computerised highfrequency trading played an essential role, and during which stock indices and important industry stocks collapsed.40 In this case ‘circuit breakers’ established by monitoring authorities set in, suspending high-speed trading and preventing further avalanche effects. These oversight and intervention mechanisms have been improved since then, but debate continues as to whether they are sufficient to prevent another significant flash crash; minicrashes and interventions occur daily.41 During the Cold War, and even afterwards, both the US and the Soviet Union received erroneous indications of nuclear attack on multiple occasions.42 These varied from sunlight reflected off clouds to magnetic training tapes fed into the early-warning system by accident. In all these cases, human reasoning led to restraint instead of escalation; double checks revealed that the alarm had been false. At the time, double checking and reconsideration were possible due to flight times of between several hours (in the case of bombers and cruise missiles) and 10–30 minutes (for ballistic missiles launched from submarines or those covering intercontinental ranges), as well as systems for preventing unwanted crisis escalation, such as the ‘hotline’ for communication between Moscow and Washington established after the Cuban Missile Crisis. Humans, or rapid-reaction mechanisms preprogrammed by humans, can also act as a fail-safe in instances where an overarching authority exists to enforce a shared set of rules, as in the stockexchange example – unlike in international politics. With the goal of improved military effectiveness providing a strong incentive to increase operational speeds, and thus to allow AWS to operate without further human intervention, tried and tested mechanisms for double-checking and reconsideration that allow humans to function as fail-safes or circuit-breakers are discarded. This, in combination with unforeseeable algorithm interactions producing unforeseeable military outcomes, increases crisis instability and is unpleasantly reminiscent of Cold War scenarios of accidental war. Setting aside the increasing risk of unwanted escalation, AWS are also bound to introduce stronger incentives for premeditated (including surprise) attacks. This is because of a combination of three factors: casualty avoidance, cost reduction and, once again, swarming. Firstly, unmanned systems, generally speaking, keep soldiers out of harm’s way – which is positive, but which also reduces the political risk of military endeavours, especially in democracies.43 Referring to the current generation of combat drones, Christof Heyns, the United Nations Special Rapporteur on extrajudicial, summary or arbitrary executions, put it this way: ‘[Drones] make it easier for States to deploy deadly and targeted force on the territories of other States.’44 As unmanned systems become faster and smaller, as well as, eventually, autonomous – which will also make them stealthier due to radio silence, and allow them to become ‘swarmier’ – the resulting room for manoeuvre in political and military terms increases. Secondly, the example of Perdix demonstrates that AWS need not be big, costly or high-tech. Instead, such systems can be cheap and disposable, produced using 3D printers and gaining strength from numbers, their ‘intelligence’ residing in a distributed fashion in the swarm or, if external communication is an option, at some higher level within the military ‘system of systems’ at large. A closely related third consideration is that swarms would make mounting a successful defence especially difficult due to their resilience and their ability to attack from many directions, simultaneously, in an overwhelming fashion. Small and very small AWS (those measuring tens of centimetres at most) would suffer from limited power supply on board, but could be brought closer to the target by riding along on ‘motherships’, as has been demonstrated with Perdix. With payloads weighing a few hundred grams at most, the amount of destructive power of small drones would be limited too. But if directed at political or military leaders or sensitive military infrastructure, they would produce relevant damage and provide entirely new means for carrying out assassinations and decapitation strikes.45 None of these points in isolation would introduce a radically novel element to military decision-making. After all, the fact that a weapon is cheap does not necessarily render it more likely to be used.46 However, the combination of these three factors – brought about mainly by the development of hard-to-defend-against autonomous swarms – presents a strong incentive to seize the advantage of being the first on the offensive. Considering the current climate between Russia and NATO, it stands to reason that old mechanisms of threat perception and worst-case thinking might see a comeback in the wake of AWS deployment.47 Russia was reportedly alarmed when the idea of using stealthy drones for missile defence was floated in the US.48 Swarms of AWS could be used to attack nuclear-weapon delivery systems, command and control systems, and sensitive infrastructure components such as antennas, sensors or air intakes. Even though an attacker might have little interest or confidence in the success of a disarming first strike of this type, the fact that such strikes were now possible would in itself increase nervousness and distrust between nuclear-armed adversaries. This overlap between the conventional and the nuclear realm is not new, of course. It emerged with precision munitions and bunker-busting (or possibly electromagnetic-pulse) warheads during the 1990s and 2000s,49 and is also documented in the New START treaty, the preamble of which states that the US and Russia are ‘mindful of the impact of conventionally armed ICBMs and SLBMs on strategic stability’.50 But AWS will likely perpetuate and intensify this trend, not least by opening up new possibilities for holding nuclear submarines carrying ballistic missiles at risk.51 Thus, when nuclear weapons or strategic command and control systems are, or are perceived to be, at greater risk, conventional capabilities end up increasing instability at the strategic level. Today’s unmanned systems have already increased the risk that military force will be used in scenarios where manned systems would previously have presented decision-makers with bigger, caution-inducing hurdles – a connection recently confirmed in war-gaming exercises.52 Of course, swarming AWS need not necessarily lead to escalation under all conditions. In asymmetric scenarios involving adversaries who lack AWS capabilities, the escalatory mechanisms developed above would not take effect. In symmetric settings, by contrast, they would certainly exacerbate the overall development toward an increased risk of crisis instability and escalation.

Sign in to recut, update, or save this card.