Free beta access. 50 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Extend – Human Control Solvency

UN action through the CCW would establish the human control norm and reduce the instability and escalation from AI weapons

International Panel on the Regulation of Autonomous Weapons, 2021 [(iPRAW) coordinated by: German Institute for International and Security Affairs, July “Building Blocks for a Regulation on LAWS and Human Control Updated Recommendations to the GGE on LAWS” https://www.readkong.com/page/building-blocks-for-a-regulation-on-laws-and-human-control-8617434 Acc 2/27/22 TA]

iPRAW recommends that the principle of human control should be internationally recognized within the CCW and possibly other documents of international law and be the basis from which requirements can be developed as part of a norm-shaping process. The elements presented below could be helpful to shape a regulation, be it legally binding or not. What is important though is to create a normative framework and operational guidance around the (development and) use of weapon systems with autonomous functions. For example, the ICRC presented one approach to such a framework by calling for a prohibition of unpredictable and anti-personnel AWS and a regulation of other AWS that considers the specific operational context. As iPRAW’s scenario-based discussions about the adequate type and level of human control illustrated, a definition of human control that adequately considers the operational context requires many details about technical capabilities and indicators for the targets. Hence, a ‘one-size-of-control-fits-all’ solution does not exist. Rather, a combination of minimum requirements and individual solutions is necessary. Individual solutions based on a case-by-case assessment will ultimately lead to different levels of granularity when it comes to formulating human control in a regulatory framework, e.g. a rather abstract declaration or treaty and more granular best practices and manuals. These could inform a regulation of LAWS and human control by creating the normative baseline and align with recommendations from other actors. As for example the ICRC discussed, it is crucial to link specific regulatory elements to the challenges raised by LAWS. Within the CCW such a regulation could cover:  military considerations: fulfilling the operational objective and translating the commanders intent to the battlefield,  legal concerns: abide by IHL principles, especially the principle of precaution by avoiding unpredictable effects, ensure human judgment to take legal decisions,  ethical concerns: retain moral agency. A CCW regulation entailing an obligation to maintain human control may also mitigate security challenges, such as conflict escalation, even though the CCW does not address them explicitly. Further important aspects, such as technology diffusion, will most likely not be addressed in a future regulation on LAWS but would have to be addressed in other fora. iPRAW’s model discussed below is not meant to read in opposition to the ICRC propositions on a prohibition and regulation of AWS but rather as an additional perspective with the same objective to keep human control in the use of force.

A UN mandate for Human Control would effectively ban LAWs – it sets the stage for future treaties.

International Panel on the Regulation of Autonomous Weapons, 2021 [(iPRAW) coordinated by: German Institute for International and Security Affairs, July “Building Blocks for a Regulation on LAWS and Human Control Updated Recommendations to the GGE on LAWS” https://www.readkong.com/page/building-blocks-for-a-regulation-on-laws-and-human-control-8617434 Acc 2/27/22 TA]

General Obligations: A regulation of LAWS, e.g. a CCW Protocol, could consist of a general obligation to maintain human control over the use of force when deploying conventional weapons. The GGE Guiding Principles adopted in 2019 could lay the groundwork to further shape a future regulation. The Guiding Principles emphasize that IHL continues to fully apply to LAWS. Furthermore, they stress that human responsibility for the decision to use LAWS must be retained since accountability cannot be transferred to machines. This aspect should be considered throughout the entire life-cycle of a LAWS. They also stipulate that human-machine interaction must be in compliance with international law and refer, among others, to questions relating to accountability, weapon reviews, and risk assessments, including the development stage of weapon systems. In addition, other principles that were not explicitly mentioned in the Guiding Principles but found entry into other documents adopted by the GGE could also supplement a future regulation. Examples are the principle of predictability, reliability, and transparency. Specific Obligations: Specific obligations and more nuanced rules on the concept of human control could play a pivotal role in a future treaty focusing on human control. It could entail concrete rules stipulating that human control encompasses both situational understanding and the option to intervene, enabled by design and in use. The term situational understanding could be elaborated in more detail by stipulating that it refers to the ability to monitor information about the environment, the target and the weapon system. The human operator shall monitor the system and the operational environment to the extent necessary in a specific operation. Furthermore, the different modes of operation should allow the human operator to intervene if necessary. All people in the chain of command are equally obliged to abide by the rules of international law and should be held accountable for any violations of the law. These and other obligations could be an integral part of “specific obligations” in a future treaty on LAWS. The ICRC mentions two specifics that would call for tighter restrictions, namely AWS with unpredictable effects and the anti-personnel use of AWS.

Sign in to recut, update, or save this card.