The area of AI excludes semi-autonomous weapons. Prefer DODD 3000.09, which is the only pertinent legal guidance published by any military on the planet. Any other interp opens the floodgates to arms sales affs for any weapon with an autonomous sensor.
Allen ’22 [Gregory; June 6; director of the Artificial Intelligence (AI) Governance Project and a senior fellow in the Strategic Technologies Program at the Center for Strategic and International Studies in Washington, D.C; CSIS; “DOD Is Updating Its Decade-Old Autonomous Weapons Policy, but Confusion Remains Widespread,” https://www.csis.org/analysis/dod-updating-its-decade-old-autonomous-weapons-policy-confusion-remains-widespread]
While describing a military system as AI-enabled or machine learning-enabled provides useful information, it often remains unclear just what functionality is being provided by AI and how central AI is to the system overall. Suppose, hypothetically, that the F-35 Joint Strike Fighter Program Office decided that it should use machine learning to improve the performance on one of the aircraft’s many different types of sensors. Does that mean that the entire F-35 program, with its more than $8.5 billion in annual spending, is building AI-enabled weapon systems? The DODD 3000.09 update is a good opportunity to formally define “AI-enabled” in DOD policy and to specify how using machine learning does or does not affect the autonomous weapon senior review process.
Confusingly, common usage of these terms differs significantly in other countries. For example, Russian weapons manufacturers routinely refer to their automated and robotic military systems as using AI even if the system does not use machine learning, and they rarely make it clear when it does. Varied definitions complicate international diplomacy on these subjects.
Define how retraining machine learning models will be handled in the senior review process.
A 2019 report from the Defense Innovation Board was memorably named “Software Is Never Done.” Updates, bug fixes, cybersecurity patches, and new features are a critical aspect of managing all modern software systems. However, software updates for machine learning systems are different in nature from traditional software, and perhaps even more important. The “learning” in a machine learning system comes from applying a learning algorithm to a training data set in order to produce a machine learning model, which is the software that performs the desired function. To update the machine learning model, the developers provide a new (usually larger) training data set that more accurately reflects the changing operational environment. If the operational environment changes, failing to rapidly retrain a machine learning model can lead to reduced performance.
Does each software update and machine learning model retraining require an autonomous weapon system to go through the senior review approval process all over again? If so, an already formidable process barrier risks unintentionally becoming an insurmountable one. A more reasonable solution would be for the senior review process to assess and decide upon the acceptability of the process for producing new training data; to test and evaluate procedures for determining whether new software updates are sufficiently robust and reliable; and to explicitly state which potential new features would trigger an additional round of the senior review approval process. This is similar to how the global automotive industry regulates software updates and safety compliance.
Clarify the specific features that formally define an autonomous weapon system.
Despite eight years of negotiations at the United Nations, there is still no internationally agreed upon definition of autonomous weapons or lethal autonomous weapons. DODD 3000.09 does, however, formally define autonomous weapon systems and semi-autonomous weapon systems from the perspective of U.S. policy:
Autonomous weapon system: A weapon system that, once activated, can select and engage targets without further intervention by a human operator. This includes human-supervised autonomous weapon systems that are designed to allow human operators to override operation of the weapon system, but can select and engage targets without further human input after activation.
Semi-autonomous weapon system: A weapon system that, once activated, is intended to only engage individual targets or specific target groups that have been selected by a human operator. This includes:
Semi-autonomous weapon systems that employ autonomy for engagement-related functions including, but not limited to, acquiring, tracking, and identifying potential targets; cueing potential targets to human operators; prioritizing selected targets; timing of when to fire; or providing terminal guidance to home in on selected targets, provided that human control is retained over the decision to select individual targets and specific target groups for engagement.
‘Fire and forget’ or lock-on-after-launch homing munitions that rely on [Tactics, Techniques, and Procedures] to maximize the probability that the only targets within the seeker’s acquisition basket when the seeker activates are those individual targets or specific target groups that have been selected by a human operator.
Notably, machine learning is not mentioned anywhere in these definitions, and that is because machine learning is not always necessary. Traditional software is sufficient to deliver a high degree of autonomy for some military applications. For example, the Israeli Aerospace Industries (IAI) Harpy is a decades-old unmanned drone that IAI openly acknowledges is an autonomous weapon. When in autonomous mode, the Harpy loiters over a given region for up to nine hours, waiting to detect electromagnetic emissions consistent with an onboard library of enemy radar, homes in on the emissions source (usually enemy air defense radar), and attacks. No human in the loop is required.
The critical feature of this definition is that the policy threshold for autonomy relates specifically and exclusively to the ability to autonomously “select and engage targets.” DOD officials talk often about their goals for increasing autonomy in weapon systems, but sometimes they mean increasing weapon autonomy in areas like guidance and navigation. For example, Mike E. White, assistant director for the DOD’s hypersonics office, stated in 2020 that he wanted future DOD hypersonic weapons to use autonomy and AI to “optimize flight characteristics.” Increasing hypersonic weapon autonomy for guidance and navigation is irrelevant from a DODD 3000.09 perspective and relatively innocuous from an international relations perspective, but the DOD should seek to minimize potential misunderstandings—especially about hypersonic weapons that could carry nuclear warheads and have significant implications for global strategic stability. This is an area where the DOD must communicate clearly and precisely.
Clarify the types of weapons that are and are not required to go through the autonomous weapon senior review process.
As mentioned above, DODD 3000.09 does not ban autonomous weapons systems but rather subjects them to additional technical and process scrutiny in the form of a senior review. However, not all weapons systems with autonomous functionality are required to go through the review. The below diagram provides a simplified depiction of what systems must go through the review and which ones are exempted. DOD should publish a similar diagram in a policy handbook to accompany the DODD 3000.09 update.