b. On probability---dwarfs the risks from nuke war, climate change, AND pandemics COMBINED.
Toby Ord 20, senior research fellow in philosophy at Oxford University and world-renowned risk-assessment expert who has advised WHO, the World Bank, the WEF, and the US National Intelligence Council, 3/3/2020, The Precipice: Existential Risk and the Future of Humanity, pp. 194-196, cc
The numbers represent my overall degrees of belief that each of the catastrophes will befall us this century. This means they aren’t simply an encapsulation of the information and argumentation in the chapters on the risks. Instead, they rely on an accumulation of knowledge and judgment on each risk that goes beyond what can be distilled into a few pages. They are not in any way a final word, but are a concise summary of all I know about the risk landscape.
Existential catastrophe via: Asteroid or comet impact Chance within next 100 years: ∼ 1 in 1,000,000
Existential catastrophe via: Supervolcanic eruption Chance within next 100 years: ∼ 1 in 10,000
Existential catastrophe via: Stellar explosion Chance within next 100 years: ∼ 1 in 1,000,000,000
Existential catastrophe via: Total natural risk Chance within next 100 years: ∼ 1 in 10,000
Existential catastrophe via: Nuclear war Chance within next 100 years: ∼ 1 in 1,000
Existential catastrophe via: Climate change Chance within next 100 years: ∼ 1 in 1,000
Existential catastrophe via: Other environmental damage Chance within next 100 years: ∼ 1 in 1,000
Existential catastrophe via: “Naturally” arising pandemics Chance within next 100 years: ∼ 1 in 10,000
Existential catastrophe via: Engineered pandemics Chance within next 100 years: ∼ 1 in 30
Existential catastrophe via: Unaligned artificial intelligence Chance within next 100 years: ∼ 1 in 10
TABLE 6.1 My best estimates for the chance of an existential catastrophe from each of these sources occurring at some point in the next 100 years (when the catastrophe has delayed effects, like climate change, I’m talking about the point of no return coming within 100 years). There is significant uncertainty remaining in these estimates and they should be treated as representing the right order of magnitude—each could easily be a factor of 3 higher or lower. Note that the numbers don’t quite add up: both because doing so would create a false feeling of precision and for subtle reasons covered in the section on “Combining Risks.”
One of the most striking features of this risk landscape is how widely the probabilities vary between different risks. Some are a million times more likely than others, and few share even the same order of magnitude. This variation occurs between the classes of risk too: I estimate anthropogenic risks to be more than 1,000 times more likely than natural risks.6 And within anthropogenic risks, I estimate the risks from future technologies to be roughly 100 times larger than those of existing ones, giving a substantial escalation in risk from Chapter 3 to 4 to 5.
Such variation may initially be surprising, but it is remarkably common in science to find distributions like this spanning many orders of magnitude, where the top outliers make up most of the total. This variation makes it extremely important to prioritize our efforts on the right risks. And it also makes our estimate of the total risk very sensitive to the estimates of the top few risks (which are among the least well understood). So getting better understanding and estimates for those becomes a key priority.
In my view, the greatest risk to humanity’s potential in the next hundred years comes from unaligned artificial intelligence, which I put at one in ten. One might be surprised to see such a high number for such a speculative risk, so it warrants some explanation.
A common approach to estimating the chance of an unprecedented event with earth-shaking consequences is to take a skeptical stance: to start with an extremely small probability and only raise it from there when a large amount of hard evidence is presented. But I disagree. Instead, I think the right method is to start with a probability that reflects our overall impressions, then adjust this in light of the scientific evidence.7 When there is a lot of evidence, these approaches converge. But when there isn’t, the starting point can matter.
In the case of artificial intelligence, everyone agrees the evidence and arguments are far from watertight, but the question is where does this leave us? Very roughly, my approach is to start with the overall view of the expert community that there is something like a one in two chance that AI agents capable of outperforming humans in almost every task will be developed in the coming century. And conditional on that happening, we shouldn’t be shocked if these agents that outperform us across the board were to inherit our future. Especially if when looking into the details, we see great challenges in aligning these agents with our values.
Some of my colleagues give higher chances than me, and some lower. But for many purposes our numbers are similar. Suppose you were more skeptical of the risk and thought it to be one in 100. From an informational perspective, that is actually not so far apart: it doesn’t take all that much evidence to shift someone from one to the other. And it might not be that far apart in terms of practical action either—an existential risk of either probability would be a key global priority.