Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Requiring Human Control is a trap – it stops our resistance to the Distancing and Elevation of modern warfare.

Kalpozos, 2020 - Prof of Law at Harvard [Ioannis 3-16-2020, , Leiden Journal of International Law, “Double elevation: Autonomous weapons and the search for an irreducible law of war", https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3545332 accessed on 6-18-2022]

‘Fully autonomous systems’ then, in relation to war-fighting, are systems that, once deployed, are able to adapt, receive, and process feedback, and display a level of functional autonomy that effectively does not distinguish them from human decision makers.24 If anything, in fact – and that is really the point – such systems may be capable of a higher level of tactical or even strategic decision-making (cognitive) and war-fighting (kinetic) capacity. This seems to be a shared understanding among states and NGOs, otherwise holding different positions in the autonomous weapons debate. Accordingly, the US Department of Defense refers to ‘[a] weapons system that, once activated, can select and engage targets without further intervention by a human operator’.25 Human Rights Watch, while setting out its position against autonomous weapons systems, defines them as ‘[r]obots that are capable of selecting targets and delivering force without any human input or interaction’.26 Other states27 and organizations28 provide similar definitions. These functions of autonomy, and their escalation, entail both physical and cognitive distancing from human agents in the overall process of targeting. Although the concept and image of autonomy dominates the discourse in a way that influences, as we will see in the next section, much of the scientific research and development, it only partially describes technological escalation. In fact, the artificial intelligence of war-fighting complements increasing autonomy with what has been called ‘merged heteronomy’. Increasing cognitive autonomy, and distance, coexists with increasingly close physical proximity. This serves pragmatic aims: Technological limitations in the fragmented development of different aspects of autonomy, the continuing necessity of human input, and the fragility of human/machine networks mean that the individual’s continuing presence in the loop remains an operational necessity. This also means that a full-on confrontation with social and political resistance to the reality of distinct killer robots is placed in abeyance. Continuous, if vague, assurances of the human remaining in the loop and retaining meaningful control are thereby facilitated. And yet, the abeyance is a trap and our presence ‘in the loop’ is no guarantee. To the extent that ‘full autonomy’ is understood to require the separate physical existence of an, often anthropomorphized, robot other, it obscures the crucial role that increasing cognitive autonomy plays in a nominally heteronomous decision-making process. Cognitively, as well as physically and kinetically, humans and machines become decreasingly separate, less and less other. Their understanding of the rules, the nomos, is merging, as is their physical existence. Increasing autonomy and merged heteronomy serve the same purpose, the same teleology of mechanization. ‘The loop’, itself, is changing, increasingly relying on artificial intelligence.

Sign in to recut, update, or save this card.