Free beta access. 50 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

“Extinction first” framing justifies genocide. Prioritize ethics.

Torres, PhD candidate in philosophy at Leibniz Universität Hannover, 21

Emile P. Torres, Aeon, Against longtermism, . October 19,2021. DOA: 06/19/23. AR.

This way of seeing the world, of assessing the badness of AIDS and the Holocaust, implies that future disasters of the same (non-existential) scope and intensity should also be categorised as ‘mere ripples’. If they don’t pose a direct existential risk, then we ought not to worry much about them, however tragic they might be to individuals. As Bostrom wrote in 2003, ‘priority number one, two, three and four should … be to reduce existential risk.’ He reiterated this several years later in arguing that we mustn’t ‘fritter … away’ our finite resources on ‘feel-good projects of suboptimal efficacy’ such as alleviating global poverty and reducing animal suffering, since neither threatens our longterm potential, and our longterm potential is what really matters.

Ord echoes these views in arguing that, of all the problems facing humanity, our ‘first great task … is to reach a place of safety – a place where existential risk’ – as he defines it – ‘is low and stays low’, which he dubs ‘existential security’. More than anything else, what matters is doing everything necessary to ‘preserve’ and ‘protect’ our potential by ‘extracting ourselves from immediate danger’ and devising robust ‘safeguards that will defend humanity from dangers over the longterm future, so that it becomes impossible to fail.’ Although Ord gives a nod to climate change, he also claims – based on a dubious methodology – that the chance of climate change causing an existential catastrophe is only ∼1 in 1,000, which is a whole two orders of magnitude lower than the probability of superintelligent machines destroying humanity this century, according to Ord.

What’s really notable here is that the central concern isn’t the effect of the climate catastrophe on actual people around the world (remember, in the grand scheme, this would be, in Bostrom’s words, a ‘small misstep for mankind’) but the slim possibility that, as Ord puts it in The Precipice, this catastrophe ‘poses a risk of an unrecoverable collapse of civilisation or even the complete extinction of humanity’. Again, the harms caused to actual people (especially those in the Global South) might be significant in absolute terms, but when compared to the ‘vastness’ and ‘glory’ of our longterm potential in the cosmos, they hardly even register.

Yet the implications of longtermism are far more worrisome. If our top four priorities are to avoid an existential catastrophe – ie, to fulfil ‘our potential’ – then what’s not on the table for making this happen? Consider Thomas Nagel’s comment about how the notion of what we might call the ‘greater good’ has been used to ‘justify’ certain atrocities (eg, during war). If the ends ‘justify’ the means, he argues, and the ends are thought to be sufficiently large (eg, national security), then this ‘can be brought to bear to ease the consciences of those responsible for a certain number of charred babies’. Now imagine what might be ‘justified’ if the ‘greater good’ isn’t national security but the cosmic potential of Earth-originating intelligent life over the coming trillions of years? During the Second World War, 40 million civilians perished, but compare this number to the 1054 or more people (in Bostrom’s estimate) who could come to exist if we can avoid an existential catastrophe. What shouldn’t we do to ‘protect’ and ‘preserve’ this potential? To ensure that these unborn people come to exist? What means can’t be ‘justified’ by this cosmically significant moral end?

Bostrom himself argued that we should seriously consider establishing a global, invasive surveillance system that monitors every person on the planet in realtime, to amplify the ‘capacities for preventive policing’ (eg, to prevent omnicidal terrorist attacks that could devastate civilisation). Elsewhere, he’s written that states should use preemptive violence/war to avoid existential catastrophes, and argued that saving billions of actual people is the moral equivalent of reducing existential risk by utterly minuscule amounts. In his words, even if there is ‘a mere 1 per cent chance’ of 1054 people existing in the future, then ‘the expected value of reducing existential risk by a mere one billionth of one billionth of one percentage point is worth 100 billion times as much as a billion human lives.’ Such fanaticism – a word that some longtermists embrace – has led a growing number of critics to worry about what might happen if political leaders in the real world were to take Bostrom’s view seriously. To quote the mathematical statistician Olle Häggström, who – perplexingly – tends otherwise to speak favourably of longtermism:

I feel extremely uneasy about the prospect that [the calculations above] might become recognised among politicians and decision-makers as a guide to policy worth taking literally. It is simply too reminiscent of the old saying ‘If you want to make an omelette, you must be willing to break a few eggs,’ which has typically been used to explain that a bit of genocide or so might be a good thing, if it can contribute to the goal of creating a future utopia. Imagine a situation where the head of the CIA explains to the US president that they have credible evidence that somewhere in Germany, there is a lunatic who is working on a doomsday weapon and intends to use it to wipe out humanity, and that this lunatic has a one-in-a-million chance of succeeding. They have no further information on the identity or whereabouts of this lunatic. If the president has taken Bostrom’s argument to heart, and if he knows how to do the arithmetic, he may conclude that it is worthwhile conducting a full-scale nuclear assault on Germany to kill every single person within its borders.

Sign in to recut, update, or save this card.