Free beta access. 50 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Circumvention is easy---there is no difference between highly autonomized systems and completely autonomous systems.

Anderson et al., 14 [Kenneth Anderson, Daniel Reisner, and Matthew Waxman, 2014, accessed on 6-27-2022,90 INT’L L. STUD. 386, Stockton Center for the Study of International Law, "Adapting the Law of Armed Conflict to Autonomous Weapon Systems", https://apps.dtic.mil/sti/pdfs/ADA613290.pdf]/ISEE

Before addressing some of the enforceability problems and dangers of any effort to prohibit autonomous weapons, it is important to understand the proposed formulas do not, as it may seem initially, contain a bright line that would be useful for promoting adherence. Lawyers experienced in the law of armed conflict will quickly see that each of these seemingly clear-cut definitions leaves many open questions as to what systems would be banned under any particular formulation. Even something as seemingly plain as “lethal decision making” by a machine does not address, among other things, the lawfulness of targeting a tank, ship or aircraft which is ultimately the source of the threat, but inside of which is a human combatant. Beyond definitions, the technology and basic architecture of an autonomous system and a nearly autonomous, highly automated system are basically the same—if you can build a system that is nearly autonomous, for example with human override, then you can probably reprogram it to eliminate that human role. Moreover, whether a highly automated system—say, one with a human supervisor who can override proposed firing decisions—is in practice operating autonomously depends on how it is being manned, how operators are trained and how effectively oversight is exercised. It also depends on operational context and conditions, which may limit the degree to which the human role is in any way meaningful. For these and other reasons, a fully autonomous system and a merely highly-automated system will be virtually indistinguishable to an observer without knowing a lot about how that system is used in particular operational conditions. The difference might not matter very much in practice, given the variable performance of human operators. In any case, these systems will be easily transitioned from one to the other. The blurriness of these lines means that it will be very difficult to draw and enforce prohibitions on “fully” autonomous systems or mandates for minimum levels of human decision making. Given the great practical difficulty of distinguishing between autonomous and highly automated systems, applying a legal ban on autonomous systems would be relatively easy to circumvent and very difficult to enforce. At the same time, and as alluded to above, imposing a general ban on autonomous systems could carry some highly unfavorable consequences— and possibly dangers. These could include providing a clear advantage in autonomous weapon technology to those States which generally would not join (or in reality comply with) such a ban. They could also include losing out on the numerous potential advantages of autonomous systems of improving decision making on the battlefield, including through avoiding emotion-based response; improving system accuracy, thereby probably minimizing collateral injuries; and possibly limiting human loss of life on both sides and among civilians.

Sign in to recut, update, or save this card.