Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

The AI Arms Race will cause miscalculation and accidents – machine speed removes checks on escalation

Sharre, 2018 - director of the technology and national security program at the Center for a New American Security [Paul, 12 September, “Ultrafast computing is critical to modern warfare. But it also ensures a lot could go very wrong, very quickly.” https://foreignpolicy.com/2018/09/12/a-million-mistakes-a-second-future-of-war/ ST]

Militaries around the globe are racing to build ever more autonomous drones, missiles, and cyberweapons. Greater autonomy allows for faster reactions on the battlefield, an advantage that is as powerful today as it was 2,500 years ago when Sun Tzu wrote, “Speed is the essence of war.” Today’s intelligent machines can react at superhuman speeds. Modern Chinese military academics have speculated about a coming “battlefield singularity,” in which the pace of combat eclipses human decision-making. The consequences of humans ceding effective control over what happens in war would be profound and the effects potentially catastrophic. While the competitive advantages to be gained from letting machines run the battlefield are clear, the risks would be grave: Accidents could cause conflicts to spiral out of control. Consider what has already happened with stock markets, where computers use algorithms to make decisions so quickly that microseconds make a difference of millions of dollars. Such trading has made brokers huge amounts of money—but has also produced extreme flash crashes that can send markets tumbling in minutes. Regulators have managed these risks by installing circuit breakers that can take a stock offline if the price moves too quickly, but battlefields lack these fail-safes. Flash crashes are bad enough; a flash war would be downright disastrous. Humans have already ceded control to machines in certain military domains. At least 30 countries—with Israel, Russia, and the United States leading the pack—employ human-supervised autonomous weapons to defend bases, vehicles, and ships. These weapons systems, such as the ship-based Aegis combat system, can detect incoming rockets and missiles and, if human supervisors do nothing, respond on their own by firing to eliminate the threat. Such automated responses allow the systems to defend against what are known as saturation attacks, in which salvos of missiles or rockets are launched at a target with such little notice that they could overwhelm human operators. For the time being, autonomous weapons such as these are used purely to protect human-occupied installations or vehicles. Humans supervise the weapons’ operation in real time and can intervene if necessary. Future autonomous weapons could lack these safeguards, however. A number of advanced militaries—including those of China, France, Israel, Russia, the United Kingdom, and the United States—are currently developing stealth combat drones intended to penetrate an adversary’s airspace. Once deep behind enemy lines, these drones might find their communications jammed, so they’re being designed to ensure they can continue to operate on their own. Most countries have not explained how their drones will operate under such circumstances and what rules of engagement they will follow. Countries could require their drones to get human authorization before launching any attacks. Doing so would allow the drones to bomb preapproved fixed targets but would require them to report back and get permission before attacking any newly discovered quarries. Such an approach sounds good in theory, but the problem is that these days many high-priority targets, such as air defense systems and ballistic missile launchers, are highly mobile. This mobility will increasingly tempt military planners to delegate lethal decision-making authority to machines, since doing so could give them an edge in reaction time. No battlefield is static, and the ability to rapidly react to a dynamic environment is critical to mission success—whether in the air, on the ground, or in cyberspace. Air combat strategists call this the OODA (Observe, Orient, Decide, and Act) loop in dogfighting. In the OODA loop paradigm of combat, pilots win dogfights not simply because they enjoy the best hardware but because they assess and react to their situations faster than their opponents, although better sensors and maneuverability might help shorten reaction times. Since machines can react faster than humans, automation will offer tremendous advantages in this competition. That means that the same competitive pressures that led to the creation of systems such as the Aegis could soon be introduced on a wider scale. A military that fully integrated its autonomous systems could always stay one step ahead of its enemy in combat and present a constantly shifting threat. As Gen. Paul Selva, the vice chairman of the U.S. Joint Chiefs of Staff, told the Senate Armed Services Committee in July 2017, “It is very compelling when one looks at the capabilities that artificial intelligence can bring to the speed and accuracy of command and control and the capabilities that advanced robotics might bring to a complex battlespace, particularly machine-to-machine interaction in space and cyberspace, where speed is of the essence.” Yet speed is not an unadulterated good. Forces that react to the enemy so quickly that their own commander does not understand what is happening could risk a breakdown in command and control, a problem that military leaders have struggled with for millennia. Today, email and chat messaging have replaced horses and flags, but the fundamental problem persists. Militaries counter this inherent friction between orders from above and the reality on the ground with a concept known as commander’s intent: succinct goal-oriented statements issued to subordinates that explain the desired goal of a particular mission, and thus ensure that they stick to the general plan, but also allow them the flexibility to adapt to events on the ground. Such statements prevent forces from becoming too predictable and give subordinates the freedom to overcome obstacles in novel ways.

Sign in to recut, update, or save this card.