Slaughterbots and AI initaited doomsday strikes.
Alexey Turchin and David Denkenberger 05-03-2018 [Turchin is of the Science for Life Extension Foundation, Moscow, Russia, Denkenberger is of the Global Catastrophic Risk Institute (GCRI), Tennessee State University, Alliance to Feed the Earth in Disasters (ALLFED), Nashville, USA, AI & Society, 147-163 (2020), “Classification of global catastrophic risks connected with artificial intelligence,” https://link.springer.com/article/10.1007/s00146-018-0845-5#Abs1//ZW]
Slaughterbots and the dangers of a robotic army Robotic minds do not require full AGI to have some form of agency: they have goals, subgoals, and a world model, including a model of their place in the world. For example, a robotic car should predict the future situation on a road, including the consequences of its own actions. It also has a main goal—travel from A to B—which constantly results in changes to the subgoal system in the form of route creation. A combination of this type of limited intelligence with limited agency may be used to turn such systems into dangerous self-targeting weapons (Turchin and Denkenberger 2018b). Commentary on narrow AI viruses It appears that if a narrow AI virus were to affect only one of the above-listed domains, it would not result in an extinction-level catastrophe. However, it is possible that there will be many such viruses, or a multipandemic (Turchin et al. 2017), or one narrow AI that will be able to affect almost all existing computers and computerized systems. In this case, if the virus(es) were deliberately programmed to create maximum damage—which could be in a case of a military grade Narrow AI virus, like the advanced version of Stuxnet (Kushner 2013)—global catastrophe is a possible result. If the appearance of narrow AI viruses is gradual, antivirus companies may be able to prepare for them. Alternatively, humans could turn off the most vulnerable systems to avoid a global catastrophe. However, a sudden breakthrough or a synchronized surprise attack could spell doom. Failure of nuclear deterrence AI Nuclear weapons are one of the most automated weapon systems. Because they must be launched immediately, almost all decision making has been done in advance. An early warning alert starts a preprogrammed chain of events, where the high-level decision should be made in minutes, which is far from optimal for human decision-making. However, the history of nuclear near misses shows (Blair 2011) that computer mistakes have been one of the main causes, and only quick human intervention has prevented nuclear war, e.g., the actions of Stanislav Petrov in 1983 (Future of Life Institute 2016). We can imagine failure modes of accidental nuclear war resulting from failure of the nuclear weapons control system. They may be similar to the Russian “dead hand” perimeter system (Bender 2014), arising if a strategic planning AI chooses a dangerous plan to “win” a nuclear war, like a Doomsday weapon (Kahn 1959), blackmail, or a pre-emptive strike.
* I couldn’t find the entire text for this, so I left the card as is.