S-risks aren’t trump cards – probability and inevitability are relevant questions in weighing them

Next, I’d like to talk about why and how to prevent s-risks.

All plausible value systems agree that suffering, all else being equal, is undesirable. That is, everyone agrees that we have reasons to avoid suffering. S-risks are risks of massive suffering, so I hope you agree that it’s good to prevent s-risks.

However, you’re probably here because you’re interested in effective altruism. You don’t want to know whether preventing s-risks is a good thing, because there are a lot of good things you could do. You acknowledge that doing good has opportunity cost, so you’re after the most good you can do. Can preventing s-risks plausibly meet this higher bar?

This is a very complex question. To understand just how complex it is, I first want to introduce a flawed argument for focusing on reducing s-risk. (I’m not claiming that anyone has advanced such an argument about either s-risks or x-risks.) This flawed argument goes as follows.

Premise 1: The best thing to do is to prevent the worst risks

Premise 2: S-risks are the worst risks

Conclusion: The best thing to do is to prevent s-risk

I said that this argument isn’t sound. Why is that?

Before delving into this, let’s get one potential source of ambiguity out of the way. On one reading, premise 1 could be a value judgment. In this sense, it could mean that, whatever you expect to happen in the future, you think there is a specific reason to prioritize averting the worst possible outcomes. There is a lot one could say about the pros and cons as well as about the implications of such views, but this is not the sense of premise 1 I’m going to talk about. In any case, I don’t think any purely value-based reading of premise 1 suffices to get this argument off the ground. More generally, I believe that your values can give you substantial or even decisive reasons to focus on s-risk, but I’ll leave it at that.

What I want to focus on instead is that, (nearly) no matter your values, premise 1 is false. Or at least it’s false if, by “the worst risks”, we understand what we’ve talked about so far, that is, badness along the dimensions of scope and severity.

When trying to find the action with the highest ethical impact there are, of course, more relevant criteria than scope and severity of a risk. What’s missing are a risk’s probability; the tractability of preventing it; and its neglectedness. S-risks are by definition the worst risks in terms of scope and severity, but not necessarily in terms of probability, tractability, and neglectedness.

These additional criteria are clearly relevant. For example, if s-risks turned out to have probability zero, or if reducing them was completely intractable, it wouldn’t make any sense to try to reduce them.