Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Distinction between military and civilian innovation – the plan doesn’t hurt the bulwark of the DIB

Vincent Boulanin and Maaike Verbruggen 2017 “MAPPING THE DEVELOPMENT OF AUTONOMY IN WEAPON SYSTEMS” November 2017 Stockholm International peace research institute https://www.sipri.org/sites/default/files/2017-11/siprireport_mapping_the_development_of_autonomy_in_weapon_systems_1117_1.pdf

The innovation ecosystem that is driving the advance of autonomy in weapon systems is diffuse, chiefly because the technologies, academic disciplines and industry sectors involved in the development of autonomous capabilities may vary greatly depending on the type of application and systems at issue. Nevertheless, three general observations can be made. First, much of the fundamental research in the fields of AI and robotics that could feed the advance of autonomy in weapon systems is dual-use. The divergence between civilian and military innovation generally emerges towards the development end of the R&D cycle because civilian and military products often need to meet different performance criteria. Hence, should CCW delegates eventually engage in a formal discussion on the monitoring or regulation of R&D efforts that could lead to the development and production of LAWS, they should focus on the development end of the R&D cycle, as this is where the actual capabilities of LAWS will be definitively created. Attempting to monitor and control R&D at the more basic research level would be challenging from a practical perspective and possibly problematic as it could threaten civilian innovation. Second, the barriers to entry to the development of robotic systems are very low. Nearly all hardware components that might serve the development of autonomous robots are commercially available. It is even possible to acquire off-the-shelf low-cost robotic systems that feature advanced autonomous capabilities. These may be adopted, modified and weaponized by states but also, and more worryingly, by non-state actors seeking, for instance, to conduct terrorist operations. This scenario has not yet received great attention within the CCW discussions on LAWS, despite the fact it represents an imminent humanitarian risk. While it falls outside the traditional scope of the CCW, it would be prudent for the GGE to allocate some time to this issue in 2018. It could start a discussion on the options that are offered outside the CCW to control or limit the diffusion and malevolent use of key technologies. This could include discussing the possibilities offered by export control mechanisms and self-control by the industry. Third, future discussions on the development and control of autonomy in weapon systems could usefully benefit from further exchanges of experience with the civilian sector, especially companies developing safety-critical applications of autonomy (e.g. aerospace companies, carmakers and medical robot companies), considering that a number of issues that are central to discussion on LAWS have already been, or are currently being, actively addressed within the civilian sphere. These issues include the following. 1. How to define and measure autonomy? This question has been the concern of standardization and regulatory bodies for a long time. The International Organization for Standardization (ISO) and the International Electrotechnical Commission have had multiple projects aimed at generating an official definition of robot-associated terms, including ‘autonomy’ (ISO 8373:2012). In addition, the US National Highway Traffic Safety Association has adopted the Society of Automotive Engineers’ levels of autonomy for self-driving systems, ranging from complete driver control to full autonomy.110 2. How to operationalize meaningful human control? Civilian industry sectors that produce safety-critical systems (e.g. aerospace, automotive and medical robotics) are facing the same human control dilemmas as the defence sector. They too are dedicating their efforts to finding a model of the human–machine command-and-control relationship that will maximize safety. 3. How to test the safety and predictability of autonomous technologies? The commercial aerospace sector has procedures to test and verify advanced automated systems. Inviting experts from this community to talk about existing procedures would help to identify possible best practices for testing and evaluating weapons with advanced autonomous capabilities in the context of Article 36 reviews.

Sign in to recut, update, or save this card.