Longtermist views of extinction invite genocide in the name of infinite good.

The dangers of utilitarian modes of moral reasoning and the utopian promise of eternal life in paradise are well known. As Pinker writes, utopian ideologies invite genocide for two reasons. One is that they set up a pernicious utilitarian calculus. In a utopia, everyone is happy forever, so its moral value is infinite. Most of us agree that it is ethically permissible to divert a runaway trolley that threatens to kill five people onto a side track where it would kill only one. But suppose it were a hundred million lives one could save by diverting the trolley, or a billion, or—projecting into the indefinite future—infinitely many. How many people would it be permissible to sacrifice to attain that infinite good? A few million can seem like a pretty good bargain. Although for Bostrom and other longtermists, the moral value is not infinite, it is astronomical. But this is more than enough, as we have seen, to justify pushing a button that trades billions of human lives for the tiniest reductions in existential risk. To quote the philosopher Thomas Nagel, writing about utilitarianism in the context of war: “Once the door is opened to calculations of utility and national interest, the usual speculations about the future of freedom, peace, and economic prosperity can be brought to bear to ease the consciences of those responsible for a certain number of charred babies.” In the case of longtermism, the issue is not the mere national interest of a state but the Ultimate Moral Interest of the Universe itself. How many charred babies is too many when the stakes are this high? One might object at this point to the association of genocide, charred babies, and the like with longtermism. “My god, no one in 23 the longtermist community is calling for anything like this! It is simply inconceivable that anyone would!” There are several problems with this reply. First, even if the leading longtermists today would ultimately side with Häggström in rejecting extreme, violent measures to secure our future among the stars, so long as the ideology remains a “legitimate” option on the marketplace of ideas, the door is open 14 for others—long-term extremists who interpret Bostrom’s words literally—to pick it up and run. In this sense, longtermism is an information hazard.15 But second, history is full of millennialist movements that rapidly flipped from one mode of eschatological thinking to another. A movement is millennialist if it holds that our current world is replete with suffering and death but will soon “be transformed into a perfect world of justice, peace, abundance, and mutual love.” In many cases, members of such movements could not, initially, have imagined their community turning to violence. It would have been unthinkable. But when external conditions change in the right way, when new circumstances trigger something in the old ideology, a community’s tolerance for bloodshed can quickly grow. As the historian Richard Landes observes, millennialism is a dynamic phenomenon, and in the course of an apocalyptic episode, a movement can literally flip from one extreme to the other. Among the classic cases, we find the Anabaptists who, in the course of their failed millennium at Münster [in Germany] from 1533-35, went from the most radically pacifist and egalitarian of the new “Protestant” groups to a violent and authoritarian group. The Anabaptists are just one of an interminable number of examples from the dusty annals of history. For example, many readers may 24 find it surprising that the notorious Japanese doomsday cult Aum Shinrikyo actually began as a peaceful movement. Over the course of just a few years, it morphed into an active apocalyptic group that tried to initiate Armageddon by releasing the poisonous gas sarin into the Tokyo subway system in 1995. Since paradise awaits on the other side of the apocalypse, the sooner the apocalypse comes, the sooner believers will enter heaven. Longtermism is, of course, a “secular” rather than “religious” movement, although one could very plausibly describe it as a quasireligion whose central object of worship is not “God” but future value. Nonetheless, plenty of millennialist movements in recent history have been “secular,” most notably those driven by the communist ideology of Marxism-Leninism. What matters is not whether the ideology is secular or religious, but whether it contains the ingredients necessary for an “apocalyptic episode” to instigate a violent response. In this case, an apocalyptic episode might be the scenario described by Häggström: putative knowledge that there is an omnicidal actor somewhere in Germany who’s chances of success are 1 in a million. It is not difficult to imagine similar situations. All one needs is a moral commitment to Utopia and the belief that there is a greater-than-negligible chance that some actor could prevent us from reaching this prized future state. We do, after all, live in the most important moment of history—the Precipice—according to longtermists, and as the posthuman author of Bostrom’s Letter from Utopia asks: “What is Guilt in Utopia? Guilt is our knowledge that we could have created Utopia sooner.”