Focusing on Human Control ignores the distancing caused by merged human/machine intelligence – it hides the violence in Human Control, and the inability of regulation to resolve our fears.
Kalpozos, 2020 - Prof of Law at Harvard [Ioannis 3-16-2020, , Leiden Journal of International Law, “Double elevation: Autonomous weapons and the search for an irreducible law of war", https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3545332 accessed on 6-18-2022]
The major stance in opposition to technological escalation towards full autonomy is centring on the taboo of delegating life/death decisions to machines and the innate inability of machines to properly apply law. A public expression of this position is that of the Campaign to Ban Killer Robots.144 The campaign is notable in resisting both superficial techno-optimism and dangerously instrumental pragmatism. The potential power of mobilization of public opposition notwithstanding, the position presents some analytical, strategic and conceptual shortcomings. Firstly, the Campaign’s approach starts from an assumption of a fundamental ‘change of paradigm’.145 This does not always appreciate the continuing role of technology in war and its function in the double elevation described here. Indeed, I have argued that the escalation towards full autonomy and merged heteronomy represents a continuation of an existing trajectory of distancing through the elevation above one’s enemy and above oneself, albeit with a perhaps justified presentiment of an ‘avalanche’, a violent acceleration of pace, out of control. Secondly, present practice does not suggest that a prospect of successful imposition of a ban or moratorium is realistic. This is due both to the anticipated military advantage and the inscription of this process in socioeconomic structures and expectations, as reflected in the evidence of enthusiastic investment in the acceleration of this trajectory. Instead, as the analysis above has suggested, while the absolute of full autonomy (combining kinetic and cognitive elements) is kept at bay, the ground is constantly prepared. Thirdly, the primary focus on preventing full autonomy or, inevitably anthropomorphized, ‘killer robots’ is in danger of missing the target. Autonomy is complemented by increasingly merged heteronomy. As discussed in Section 2.1 above, merged heteronomy addresses the logistical limitations of spatially spread human/machine networks. Crucially, while presenting itself as respecting the moral taboo of life-and-death delegation, merged heteronomy advances the mechanization of judgement in pursuit of double elevation. To the extent that the maintenance of ‘meaningful human control’ is primarily focused on ‘keeping humans in the loop’, it is in danger of ignoring the gradual change in the nature and function of that very loop. As reliance on artificial intelligence increases, it is humans who are becoming the ‘killer robots’. Finally, to the extent that the position relies on the incompatibility of autonomous weapons with international humanitarian law, it may be vulnerable to the complicity, discussed in Section 3.3, of certain strands of legal thinking with an understanding of knowledge as ‘a large store of neutral data’146 and the promise of the piece-meal resolution of technical legal problems. It also allows one’s intuitive angst to be assuaged by promises of, or indeed steps towards, the panacea of global regulation.147 As important as such regulation may be in structuring the ambitions of both state and private actors, it would not, per se, address the most fundamental dangers of the mechanization of judgement. Law will neither ban nor regulate away what causes our angst. To the contrary, it may be adapted to serve mechanized judgement. If we are to oppose double elevation and the mechanization of judgement, and hope to use law to this effect, we need legal thinking to serve this purpose. Otherwise, all we can do is surrender to the stance of agnostic abeyance, until the code is engineered.