Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Humans in the loop cannot prevent accidents – they can only hope to limit the damage after the fact.

Scharre, 2016 -- Vice President and Director of Studies at CNAS [Paul, Feb 2016, Center for New American Security, “Autonomous weapons and operational risk”, https://s3.us-east-1.amazonaws.com/files.cnas.org/documents/CNAS_Autonomous-weapons-operational-risk.pdf?mtime=20160906080515&focal=none, Acc 6/21/22, M. A.]

Autonomous Weapons and Operational Risk 11 have a higher inherent hazard than one on a closed track, the type of human control over the car could significantly change the damage potential. A self-driving car that is equipped with a steering wheel and brake to allow the human operator to take control and stop the vehicle (humansupervised autonomy) has, in principle, lower damage potential than a fully autonomous car where the human is merely a passenger along for the ride. The speed of interactions matters significantly, however. Giving the human operator the ability to grab the wheel of an autonomous vehicle traveling at highway speeds in dense traffic, particularly if the operator is not paying attention, is merely the illusion of control. Conversely, a brake and steering wheel on an autonomous car moving slowly under the supervision of an attentive human operator might add real value by allowing the human to function as an additional fail-safe. The driver may not be able to prevent all accidents (after all, humans are not great drivers even when directly in control of the vehicle), but he or she could prevent an autonomous car from running rampant, senselessly mowing down pedestrians. An unfortunate reality of both supervised autonomous and even semi-autonomous operation is that the human operator may not become aware that the system is failing until after a failure occurs. A human in the loop or on the loop will not necessarily prevent failures from occurring. However, the ability of a human to undertake corrective action can help limit the damage potential of a system if it fails. Thus, in these circumstances, the human functions as a fail-safe. The human operator cannot necessarily prevent failure, but he or she can help ensure that if or when the system fails, the damage is limited.

No solvency – there is no agreed upon standard for Human Control yet.

Amoroso and Tamburrini, 2021 - Prof of International Law at the University of Cagliari and Prof of Philosophy of Science and Technology at the University of Naples Federico [Daniele and Guglielmo, Feb Italian Journal of International Affairs “In Search of the ‘Human Element’: International Debates on Regulating Autonomous Weapons Systems” TM]

The ‘weaponisation’ of AI and robotics, especially their convergence in autonomous weapons systems (AWS), is undoubtedly a matter of international concern. AWS academic and diplomatic debates have revolved around AWS’ distinguishing features and why their destructive force is especially troublesome from ethical and legal standpoints. Discussions about steps the international community can take to allay these normative concerns effectively have progressively been zooming in on the distinctive role the ‘human element’ plays in the use of force. Indeed, there is broad agreement that the identification of normatively acceptable human-weapon interactions must constitute the keystone of any future regulation of AWS. Both the Campaign and an increasing number of states maintain that a ruling should be established under international law that any weapons system must be subject to meaningful human control (MHC). This, however, is exactly where international consensus stops; it is far from clear, even among those favouring an MHC requirement, exactly what its actual content should be or, to put it more accurately, what is normatively demanded to make human control over weapon systems truly ‘meaningful’.

Sign in to recut, update, or save this card.