Refuse monocausal, linear accounts of extinction events in favor of a present apocalypse that is accelerated by structural failures in social systems.

1. Introduction the definition and framings of existential risk¶ In recent years, a growing body of scholarship has argued that a new class of risks bears closer study, for their potential extreme impact on the survival of humanity (Bostrom, 2002, 2013; Bostrom & Cirkovic, 2011; Matheny, 2007; Rees, 2004). Prior research has identified a range of such human extinction risks (Bostrom & Cirkovic, 2008; Haggstrom, 2016; Pamlin & Armstrong, 2015), both natural and manmade, including risks from supervolcano eruption, asteroid impact, global warming, nuclear war, as well as more speculative risks from emerging technologies such as biotechnology, high-energy physics experiment disasters, or misaligned artificial intelligence. (Asimov, 1981; Baum & Barrett, 2016; Bostrom, 2014; Ord, Hillerbrand, & Sandberg, 2010; Posner, 2004; Sagan, 1983; Smil, 2005; Tegmark & Bostrom, 2005; Yudkowsky, 2008a).¶ While it is encouraging to see greater attention for a critical topic that has long remained understudied, it is relevant to ask how the framing of the field’s basic concepts shapes both which problems it identifies and prioritizes, as well as which policy approaches it considers and engages. In his seminal paper, Bostrom defined an existential risk as ‘[o]ne where an adverse outcome would either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential’ (Bostrom, 2002). Thus in Bostrom’s view, existential risks are characterised both by their scope (pan-generational) and their intensity (crushing): the size of the group of people who are at risk1 and how badly each individual within that group is affected, respectively (Bostrom, 2002).¶ Much prior research on existential risks has thus deployed criteria and methodology which have identified discrete and independent challenges of sufficient severity and pervasiveness to bring about the ‘adverse outcome’ in a direct causal manner. In this reading, existential risks are an extreme offshoot of global catastrophic risks—disasters which “might have the potential to inflict serious damage to human well-being on a global scale” (Bostrom, 2013; Bostrom & Cirkovic, 2011, p. 2), but which fall short of permanent collapse. While we are not necessarily averse to the Bostromian definition of ‘adverse outcomes’—a definition which indeed seems to characterize the space of eventual outcomes to be avoided—we take more issue with the limited range of pathways towards this dreaded outcome-space, which much of the literature has focused on exploring. Specifically, as noted by others in the community, much prior research “has focused mainly on tracing a causal pathway from a catastrophic event to global catastrophic loss of life” (Avin et al., 2018, p. 1). As such there remains an event-focus, in the sense that only discrete events that are causally connected to the demise of humanity within a relatively short time-frame qualify as an existential risk (rather than a ‘merely’ globally catastrophic one, or a background risk).¶ 1.1. Existential risk (re)framings as crucial consideration for law & governance approaches¶ Distinguishing existential risks as a uniquely threatening outlier along the spectrum of global risks, however, is arguably an unnecessarily narrow framing of the field of study. Indeed, a high-profile ‘one-hit-KO’ existential risk such as a global nuclear war or a pandemic may constitute only one avenue towards that ‘adverse outcome’, and concentrating predominately upon (ways to intervene in) its origin and direct pathway, risks overshadowing other potential paths or disaster interaction effects2 that functionally converge towards that same disastrous outcome, even if only indirectly or over longer timescales, with a potentially higher probability. Indeed, as recently noted by scholars in the field, a full mapping of scenarios that lead to catastrophic outcomes “requires exploring the interplay between many interacting critical systems and threats, beyond the narrow study of individual scenarios that are typically addressed by single disciplines” (Avin et al., 2018, p. 2).The precise framing of ‘existential risks’ is therefore a crucial consideration, informing ethical, strategic, and epistemological (cf. academic) priorities in facing ‘adverse outcomes’. This is particularly the case in the context of studying how global political dynamics may interact with certain existential risks, and in formulating meaningfully effective policies and governance approaches to such risks. Of course, this is not to say that the field of existential risk studies has not sought to involve and engage with policy and governance approaches and solutions. Indeed, to its credit, research in the field of existential risks has actively sought to engage with these issues—given that, as Bostrom himself observes (2013:27), global cooperation is critical to mitigating a wide range of existential risks. Likewise, researchers within the ‘AI safety’ community are beginning to highlight fields such as policy and psychology, as under-represented but potentially promising approaches to addressing risks arising from AI (Brundage, 2017; Sotala, 2017b).¶ Accordingly, there has been research into the interaction effects between technologies and politics—such as the possibility that arms races might increase the risk that untested, powerful AI systems are deployed rashly or prematurely (Armstrong, Bostrom, & Shulman, 2013; Shulman, 2009). Other work has drawn on cognitive psychology, to study how people might structurally (mis)judge the probability of risks (Yudkowsky, 2008b). Likewise, work exploring policy- and governance approaches to mitigating existential risks has explored policy approaches that include insurance arrangements for large catastrophes (Taylor, 2008); technology taxes and subsidies (Posner, 2008); and work drawing on social (and organizational) psychology to assess ways to motivate AI researchers to choose beneficial AI designs (Baum, 2016). Yet, other work has examined the cost-effectiveness of biosecurity interventions (Millett & Snyder-Beattie, 2017); pricing externalities to balance public the risks and benefits of scientific research generally (Farquhar, Cotton-Barratt, & Snyder-Beattie, 2017); and proposing a general international regulatory regime to govern global catastrophic and existential risks from emerging technologies (Wilson, 2013). At present, a majority of existential risk research centres3 have articulated law and policy research as areas of interest, and scholars in this space have begun to translate such work into concrete proposed policy interventions—notably the 2017 GPP report, which included proposals to develop governance for geoengineering research; establish international scenario plans and exercises for engineered pandemics, and build international attention for existential risk reduction (Farquhar, Halstead et al., 2017).¶ Such work is highly encouraging, and the existential risk research agenda has benefited from it. Nonetheless, the risk remains that a too-narrow conception of ‘existential risks’ prematurely closes down the space of law and governance solutions that are possible—or necessary—in assuring humanity a non-catastrophic future—for instance, a future that, in Bostrom’s framing, meaningfully ‘maximize[s] the probability of an ok outcome’ (Bostrom, 2013, p. 19). However, if human extinction and the persistent and pervasive truncation of technological potential are not completely homologous, then tailoring our portfolio of policy responses exclusively to closing off the pathways these risks could take—and then calling it a day—would be insufficient. In fact, this might only afford future policymakers with a false sense of security, even as the world continues to reside in an overall state of ‘super-risk’ (Bermudez & Pardo, 2015).¶ This is especially the case when there is a narrow ‘technological’ (re)solution on offer—such as ‘improve global vaccine synthesis and production capability’, or ‘subsidize international technical AI safety research’—which promise to address or prevent the risk at its root. While such direct technological solutions may certainly be indispensable to averting some existential risks, they may not suffice in actually ‘plugging all the holes’ in our risk space. In a disciplinary context, there is a risk (admittedly self-correcting, given publication incentives) of the research agenda ‘halting’ early. In a real-world context, the availability of simple, straightforward ‘fixes’ might even pose a ‘moral hazard’, if policymakers or global governance systems which lack political will or the attention to explore more complex or costly changes, seize upon the ‘symbolic action’ of the straightforward, first-order mitigation strategies. Even where this is not the case, certain policy recommendations to mitigate existential risks might depend on too-optimistic a view of institutional rationality or capability.¶ 1.2. ‘Boring Apocalypses’: from existential hazards, to existential risks¶ While such efforts might mitigate specific existential risks, this might not translate into significantly lowering the overall probability of the ‘adverse outcome’, if only a part of the problem, or only one problem among many, is addressed. An alternative articulation is that only one path to the ‘adverse outcome’ is being explored by much research into existential risks: erecting obstacles along that path may indeed reduce the overall likelihood of manifesting these risks, but this might have little impact, or even no effect, upon the manifestation of the ‘adverse outcome’.¶ Thus, our view is that a materialised existential risk (what we call an ‘existential hazard’) is sufficient to lead to an (existentially) ‘adverse outcome’, but crucially, that this is unnecessary to reach that result. If the overarching objective is to lower the probability of human extinction or significant technological curtailment, adopting an array of approaches which complement the mitigation of direct existential risks are required. Within this broad spectrum of aligned approaches, we propose to introduce law, policy, regulatory and governance tools in this paper as an example. The choice of law and policy perspectives is two-fold: on one hand, they make it possible to take second-order considerations, which involve indirect and socially and culturally mediated paths towards ‘adverse outcomes’ into account; on the other hand, these recognise both the complexity of social organisation and the prospect that civilizational collapse may trigger or possibly instantiate existential outcomes. In this sense, law and policy approaches offer the possibility of complementing and enhancing the narrower approach adopted by contemporary existential risk research, to take into consideration other paths to existentially adverse outcomes; and to better anticipate vulnerabilities, exposures and failure modes in societal efforts to address existential risks.¶ 1.3