Each new category is existential

This section turns to an empirical analysis of the forces of destruction, based on evidence from four cases of anthropogenic existential threats: nuclear war, climate change, bioengineered pathogens, and artificial intelligence. The analysis identifies three empirical trends in the evolution of the forces of destruction (see table 2). First, there are now three modes of human agency that constitute the threat of physical destruction: violence between human actors, destruction of the natural environment, and loss of control over technology. Second, the scale of physical destruction has reached the critical point at which it becomes an existential threat to human civilization and survival—the forces of destruction have become “total.” Third, the diffusion of technology has made it so nationstates no longer possess a monopoly over the forces of destruction—non-state actors have become potential “agents of destruction.”

Modes of Destruction

The first variable—the modes of destruction—concerns the kinds of human agency that constitute the threat of physical destruction. There are essentially three modes of destruction behind anthropogenic existential [[FIGURE 1 OMITTED]] threats. While each represents a distinct form of human agency, all are instances of humanity’s capacity for self-destruction, and all are relevant for understanding the forces of destruction (see figure 1).

The first mode of destruction is violence between humans, which makes possible the threat of omnicide. This could take the form of violence by states (e.g., nuclear war) or non-state actors (e.g., bioterrorism). Arguably, the prospect of omnicidal violence is best captured by the threat of nuclear war. Since the dawn of the “nuclear revolution,”8 humans have maintained nuclear peace through a “balance of terror” between nation-states (Wohlstetter 1959). For Bernard Brodie (1959, 180), “deterrence is simply the effort to erect appropriate disincentives to counteract the incentives which the opponent feels for our destruction.” In a thermonuclear world, deterrence is based on the expectation that the outcome of nuclear war would be “mutual assured destruction” (MAD). In operational terms, nuclear deterrence relies on both sides possessing a secure and credible retaliatory nuclear capability (“second strike”) after having already been struck by a preemptive nuclear attack (“first strike”) (Brodie 1959; Wohlstetter 1959; Jervis 1989; Waltz 1990).

However, a combination of factors could erode the stability of nuclear deterrence, including the changing balance of power (e.g., US hegemonic decline, the resurgence of Russia, and the rise of China); deterioration of nuclear arms control (e.g., the demise of the Anti-Ballistic Missile Treaty and Intermediate Nuclear Forces Treaty, and uncertain future of the New START Agreement); the shift in military doctrines towards lower nuclear thresholds against conventional military force (e.g., Russia’s “escalate-to-de-escalate” doctrine, and the US 2018 Nuclear Posture Review); the sanguine rhetoric by some political leaders about nuclear weapons that undermine the “nuclear taboo” (Tannenwald 2018); and military-technological changes that could make states more vulnerable to a “disarming” first-strike (e.g., cyber and space capabilities, precision-strike and hypersonic missiles, and kinetic or direct-energy ballistic missile defence) (Gartzke and Lindsay 2017; Lieber and Press 2017; Miller and Fontaine 2017; Acton 2018). Nuclear war could always occur by accident or miscalculation (Blair and Kendall 1990). The danger is most obvious in an international crisis, such as the Cuban Missile Crisis in October 1962, when the United States and the Soviet Union narrowly avoided nuclear war; or the escalation of tensions between the United States and North Korea in 2017, when North Korea conducted ballistic missile and nuclear tests and US President Donald Trump threatened to unleash “fire and fury” on Pyongyang. There has also been a long list of “close-calls” with nuclear weapons because of human and technical vulnerabilities in nuclear command-and-control, including cases where preparations were made for nuclear war due to a malfunctioning computer chip, or early-warning systems misinterpreting natural phenomenon as incoming missile attacks (Blair 1985; Sagan 1995; Schlosser 2013).

The second mode is human destruction of the natural environment, which could leave the planet uninhabitable for humanity (Ward 2008; Brannen 2017; Steffen et al. 2018; Spratt and Dunlop 2019; Wallace-Wells 2019; Ripple et al. 2020). Humans have become the driving force behind environmental change, which is why this geological epoch is increasingly called “the Anthropocene” (Crutzen 2002), to distinguish it from the unusually temperate period since the end of the last Ice Age roughly 11,700 years ago, the Holocene, when human civilization—probably not coincidentally—began and flourished (Steffen, Crutzen, and McNeill 2007).9 The decline of some of Earth’s environmental subsystems already exceeds the “planetary boundaries” believed to constitute a “safe operating space for humanity” (e.g., climate change, biodiversity loss, and the nitrogen cycle), while others are under serious strain (e.g., ocean acidification and the phosphorus cycle). The outcome of human-driven environmental decline could be “irreversible and, in some cases, abrupt environmental change, leading to a state less conducive to human development” (Rockstrom et al. 2009, 472). The species extinction crisis—what some are calling the “sixth mass extinction” (Kolbert 2014; Brannen 2017)—is a sign of Earth’s declining habitability in the Anthropocene.10

The biggest environmental danger may be climate change (or “global warming”). The major anthropogenic drivers of climate change are the burning of fossil fuels (e.g., coal, oil, and gas), combined with deforestation for agriculture (e.g., livestock and monocultures) and resource extraction (e.g., mining and oil), plus ocean warming undermining the planet’s capacity for absorbing carbon dioxide. Anthropogenic climate change could soon pass certain “tipping points,” whereby positive feedback loops in Earth’s climate system could lead to potentially irreversible and self-reinforcing “runaway” global warming (Kump, Kasting, and Crane 2003; Steffen et al. 2018). For example, the melting of Arctic “permafrost” could produce additional warming, as glacial retreat reduces the refractory effect of the ice and releases huge quantities of methane currently trapped beneath it. Earth could then move towards a “Hothouse Earth” climate, which would make the planet a less-hospitable place for humanity (Steffen et al. 2018; Spratt and Dunlop 2019; Wallace-Wells 2019; Ripple et al. 2020).

The third mode is human development of technology, which engenders the risk of loss of control over a singularly powerful technology (Moravec 1988; Vinge 1993; Bostrom 2014; Shanahan 2015; Danzig 2018; Tegmark 2018; Russell 2019). Bostrom (2013, 25) succinctly describes the risk: “If we continually sample from the urn of possible technological discoveries . . . then we risk eventually drawing a black ball: an easy-to-make intervention that causes extremely widespread harm and against which effective defense is infeasible.” Several emerging technologies pose a loss-of-control risk, including biotechnology (e.g., the “CRISPR revolution” could give humans the power over evolution to “rewrite the code of life,” including “gene-editing” of the human germ line or large-scale “gene-drives” in nature) (Doudna and Sternberg 2017); nanotechnology (e.g., the so-called “grey goo” scenario, whereby self-replicating nanotechnology could proliferate exponentially and uncontrollably until the entire biosphere is converted into nanomachines) (Drexler 2006); and artificial intelligence (e.g., the possibility of artificial “superintelligence”)

Perhaps AI best captures the threat of technological loss of control. Artificial intelligence is broadly defined as digital technologies or computers “that are capable of performing tasks commonly thought to require intelligence” (Brundage et al. 2018, 9). Currently, progress in AI is being driven by a combination of gains in hardware (e.g., the exponential growth in computing power described by “Moore’s Law”), software (e.g., “machine learning” algorithms and techniques, such as “neural networks” and “deep learning”), and data (e.g., the abundance of digital information on the Internet) (Bostrom 2014). Today’s AI are “narrow” systems that can achieve or surpass human-level intelligence only in specific domains—as demonstrated in November 2017, when “AlphaGo” beat the world champion, Ke Jie, at the strategy game “Go.” The idea that computers could one day possess human-level “general” intelligence was first suggested by none other than Alan Turing, who described a test—the “Turing Test”—in which a computer would successfully persuade a human observer that it was human. Since then, AI experts have frequently expressed the concern that AI could one day far surpass human beings in general intelligence—i.e., “superintelligence” (Good 1966; Moravec 1988; Vinge 1993; Kurzweil 2006; Bostrom 2014; Shanahan 2015; Tegmark 2018; Russell 2019). In one scenario, an AI system pursues the goal of optimizing its own algorithm, which becomes an accelerating process of self-recurring improvements—an “intelligence explosion”—that approaches a near-vertical ascent in capabilities—the “singularity.” The essence of the “control problem” is about figuring out how to properly align the goals of AI with humanity before an “intelligence explosion”; for once superintelligence exists it could become impossible for humans to control (Bostrom 2014; Shanahan 2015; Tegmark 2018; Russell 2019).