Human dignity demands that humans remain in the loop for artificial intelligence weapon systems.
International Panel on the Regulation of Autonomous Weapons, 2021 [(iPRAW) coordinated by: German Institute for International and Security Affairs, July “Building Blocks for a Regulation on LAWS and Human Control Updated Recommendations to the GGE on LAWS” https://www.readkong.com/page/building-blocks-for-a-regulation-on-laws-and-human-control-8617434 Acc 2/27/22 TA]
The precautionary principle: The obligation to take precautions is not limited to attacks but the principle applies – at least to a certain extent – to military operations in general. The notion of “military operations” has a broader meaning encompassing “any movements, maneuvers and other activities whatsoever carried out by the armed forces with a view to combat.” Parties to a conflict have to take all feasible measures to verify the target and to avoid excessive damage to civilians. Weapon systems based on computational methods could be a helpful instrument to support humans in their decision-making processes, especially with regard to target verification. However, in case LAWS are deployed, it would be questionable whether such weapon systems are able to obtain reliable information to distinguish adequately between legitimate and illegitimate targets. As adumbrated above, the distinction between combatants and civilians and between military and civilian objects respectively is challenging, often requiring the cognitive capacity to detect slight nuances in human behavior and changes of circumstances. Furthermore, the principle of precautions in attack requires parties to a conflict to take feasible measures to avoid excessive harm to civilians. This entails the obligation to deploy weapons with predictable effects which, in turn, necessitates an adequate understanding of the weapon system and its operating principles by the human operators and the military commanders. Furthermore, the use of weapon systems based on computational methods may oblige parties to a conflict to apply even higher standards of precaution. It is also important to bear in mind that the principle of precautions requires parties to a conflict to abort a mission in case the military objective turns out to be civilian. A LAWS would have to be able to autonomously abort a mission in case of doubt regarding the respective target. One way to address the risk of legal violations posed by the employment of LAWS is to regulate how these systems are used, i.e. with human control, rather than to regulate the systems, that is, their numbers or capabilities, themselves. Human Control as a Consequence of IHL? The obligation to maintain human control arguably derives from IHL, at least by implication. However, IHL does not indicate how human control should be operationalized. It is arguable that the principles of distinction, proportionality, and precautions in attack imply a requirement for the user to have sufficient situational understanding but also options for intervention. Accordingly, the operator/commander must be able to review legal assessments and translate human decision-making into the system’s action during attack prior to the actual engagement. One option to exert control is to impose operational constraints on LAWS, for example, by not using them in anti-personnel mode or against military objects by use. Even if some of those legal issues were be solved by technological achievements in the future, certain ethical challenges would still call for human control in the use of force, especially if used against human targets. Therefore, iPRAW’s ethical considerations focus on the concept of human dignity. Based on the work of Peter Asaro and Christof Heyns, we defined a set of three minimum requirements for human dignity in the use of force as the ability to: Recognize a human being as a human, not just distinguish it from other types of objects and things but as a being with rights that deserve respect; Understand the value of life and the significance of its loss; and Reflect upon the reasons for taking life and reach a rational conclusion that killing is justified in a particular situation. Depending on the moral position, one would assume or deny that autonomous functions in weapon systems break the link to moral agency. In the first case, it would be necessary to safeguard moral agency through human control, in the latter case one would want to safeguard the ability to use a weapon system lawfully at the current state of technology. In consequence, both positions would require human control in both the design of the system and in its use. Inherent in both views is an acknowledgment – tacit or explicit – of the principle of human control.