Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Human control is based on cybernetic thinking – the merging of human/machine teams

Kalpozos, 2020 - Prof of Law at Harvard [Ioannis 3-16-2020, , Leiden Journal of International Law, “Double elevation: Autonomous weapons and the search for an irreducible law of war", https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3545332 accessed on 6-18-2022]

3.2. Rising above oneself ‘We are in an arms race with ourselves – and we are winning’89 Technological evolution in war is not only about overcoming the enemy. It is also about overcoming one’s own imperfections in the wielding of violence. It is a process of progress, improvement, rationalization, optimization, ultimately the civilization of war-fighting. The role of this second elevation, which both facilitates and aims to justify the elevation above one’s enemy, is often underappreciated. I will highlight it in this section, complementing the historical narrative above and recognizing its influence on a certain view of the relationship between technology, war and international law. Elevation above oneself does not require asymmetrical relationships. The technological impetus of air power did not only serve the purpose of offense. It played a crucial role in the development of the relationship between human and machine for defensive purposes. The efforts to counter distancing and provide an effective defence against the German Luftwaffe and the early smart bomb technology of the V-1 and V-2 missiles significantly pushed forward artificial intelligence research.90 One such effort, led by Norbert Wiener, focused on the scientific articulation of human-machine interaction and the understanding of a pilot and his aircraft as a single unit, an integrated system, the behaviour of which could be predicted. While not successfully weaponized, the research led to Wiener’s theory of cybernetics,91 a widely influential theory for the scientific understanding of information, communication and the function of individuals in their socio-technical environment. Cybernetics is crucial for the evolution of human/machine merging, and the perception of self-improvement alongside the elevation above one’s adversary. Cybernetics is especially important for re-thinking law and agency in autonomous systems as it is, at the same time, based on a formalized understanding of information as the elementary unit for any sort of communication (human/human, human/machine, machine/human, or machine/machine) while having critical implications in relation to our understanding of agency and autonomy in human/machine systems. Therefore, it can be useful in appreciating that increasing autonomy and merged heteronomy are not opposites and that a ‘human-in-the-loop’ is not, by itself, the answer to the question of mechanization of judgement.92 For cybernetics, as Peter Galison has pointed out, the enemy, the German Luftwaffe, with its smart missiles and able pilots, is already perceived as hyper-rational, an advanced unit of human/machines, ‘a mechanized Enemy Other’.93 The distance already achieved by the enemy is an impetus for understanding them, through Wiener’s research, as a merged human/machine system. This perception of the enemy and the effort to predict their behaviour extends to and corresponds to the cybernetic perception of the world, and ourselves, as merged human/machine systems. The understanding of the enemy’s humanity as partial, as merged with a technical system, is reflected back to the view of oneself and it is emulated. Their distance becomes our distance; their elevation is the impetus for ours. A formalized system of information sharing and a merging in technological structures is the way forward. This is what cybernetics endeavoured to provide. Such impetus for self-improvement through military technology was applied to the creation of broader systems for the governance of war. In the Cold War, alongside the offensive asymmetry of Vietnam’s aspired automated battlefield, the period saw the creation of sophisticated human/machine systems for defensive purposes as well. The massive investment in the Semi-Automatic Ground Environment (SAGE) system in the 1950s – ‘the first large-scale, computerized command, control and communications system’ was aimed at ‘global oversight and instantaneous military response’.94 The identification of and response to the incoming threats would remain at a distance, achieved through the merging of human and machine surveillance power, in a complex and holistic system of artificial intelligence. In this mode of active defense, elevation above oneself and elevation above one’s enemy are seen as mutually reinforcing. The creation of distance and asymmetry in the elevation above one’s enemy envisions the conduct of war through an increasingly vertical relationship akin to governance.95 This entails qualities and aspirations associated with rational governance.96 Such qualities, like the rationalization and optimization of decision-making, span the range of war-making, from the level of planning and prioritizing targets (for example, on the production of ‘kill lists’97) to the level of the individual decision maker: the one pressing the button. The self-improvement through technology that puts one party in a position to govern through war is displayed in how that party governs through war, justifying its dominance.

Sign in to recut, update, or save this card.