1. Introduction the definition and framings of existential risk¶ In recent years, a growing body of scholarship has argued that a new class of risks bears closer study, for their potential extreme impact on the survival of humanity (Bostrom, 2002, 2013; Bostrom & Cirkovic, 2011; Matheny, 2007; Rees, 2004). Prior research has identified a range of such human extinction risks (Bostrom & Cirkovic, 2008; Haggstrom, 2016; Pamlin & Armstrong, 2015), both natural and manmade, including risks from supervolcano eruption, asteroid impact, global warming, nuclear war, as well as more speculative risks from emerging technologies such as biotechnology, high-energy physics experiment disasters, or misaligned artificial intelligence. (Asimov, 1981; Baum & Barrett, 2016; Bostrom, 2014; Ord, Hillerbrand, & Sandberg, 2010; Posner, 2004; Sagan, 1983; Smil, 2005; Tegmark & Bostrom, 2005; Yudkowsky, 2008a).¶ While it is encouraging to see greater attention for a critical topic that has long remained understudied, it is relevant to ask how the framing of the field’s basic concepts shapes both which problems it identifies and prioritizes, as well as which policy approaches it considers and engages. In his seminal paper, Bostrom defined an existential risk as ‘[o]ne where an adverse outcome would either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential’ (Bostrom, 2002). Thus in Bostrom’s view, existential risks are characterised both by their scope (pan-generational) and their intensity (crushing): the size of the group of people who are at risk1 and how badly each individual within that group is affected, respectively (Bostrom, 2002).¶ Much prior research on existential risks has thus deployed criteria and methodology which have identified discrete and independent challenges of sufficient severity and pervasiveness to bring about the ‘adverse outcome’ in a direct causal manner. In this reading, existential risks are an extreme offshoot of global catastrophic risks—disasters which “might have the potential to inflict serious damage to human well-being on a global scale” (Bostrom, 2013; Bostrom & Cirkovic, 2011, p. 2), but which fall short of permanent collapse. While we are not necessarily averse to the Bostromian definition of ‘adverse outcomes’—a definition which indeed seems to characterize the space of eventual outcomes to be avoided—we take more issue with the limited range of pathways towards this dreaded outcome-space, which much of the literature has focused on exploring. Specifically, as noted by others in the community, much prior research “has focused mainly on tracing a causal pathway from a catastrophic event to global catastrophic loss of life” (Avin et al., 2018, p. 1). As such there remains an event-focus, in the sense that only discrete events that are causally connected to the demise of humanity within a relatively short time-frame qualify as an existential risk (rather than a ‘merely’ globally catastrophic one, or a background risk).¶ 1.1. Existential risk (re)framings as crucial consideration for law & governance approaches¶ Distinguishing existential risks as a uniquely threatening outlier along the spectrum of global risks, however, is arguably an unnecessarily narrow framing of the field of study. Indeed, a high-profile ‘one-hit-KO’ existential risk such as a global nuclear war or a pandemic may constitute only one avenue towards that ‘adverse outcome’, and concentrating predominately upon (ways to intervene in) its origin and direct pathway, risks overshadowing other potential paths or disaster interaction effects2 that functionally converge towards that same disastrous outcome, even if only indirectly or over longer timescales, with a potentially higher probability. Indeed, as recently noted by scholars in the field, a full mapping of scenarios that lead to catastrophic outcomes “requires exploring the interplay between many interacting critical systems and threats, beyond the narrow study of individual scenarios that are typically addressed by single disciplines” (Avin et al., 2018, p. 2).The precise framing of ‘existential risks’ is therefore a crucial consideration, informing ethical, strategic, and epistemological (cf. academic) priorities in facing ‘adverse outcomes’. This is particularly the case in the context of studying how global political dynamics may interact with certain existential risks, and in formulating meaningfully effective policies and governance approaches to such risks. Of course, this is not to say that the field of existential risk studies has not sought to involve and engage with policy and governance approaches and solutions. Indeed, to its credit, research in the field of existential risks has actively sought to engage with these issues—given that, as Bostrom himself observes (2013:27), global cooperation is critical to mitigating a wide range of existential risks. Likewise, researchers within the ‘AI safety’ community are beginning to highlight fields such as policy and psychology, as under-represented but potentially promising approaches to addressing risks arising from AI (Brundage, 2017; Sotala, 2017b).¶ Accordingly, there has been research into the interaction effects between technologies and politics—such as the possibility that arms races might increase the risk that untested, powerful AI systems are deployed rashly or prematurely (Armstrong, Bostrom, & Shulman, 2013; Shulman, 2009). Other work has drawn on cognitive psychology, to study how people might structurally (mis)judge the probability of risks (Yudkowsky, 2008b). Likewise, work exploring policy- and governance approaches to mitigating existential risks has explored policy approaches that include insurance arrangements for large catastrophes (Taylor, 2008); technology taxes and subsidies (Posner, 2008); and work drawing on social (and organizational) psychology to assess ways to motivate AI researchers to choose beneficial AI designs (Baum, 2016). Yet, other work has examined the cost-effectiveness of biosecurity interventions (Millett & Snyder-Beattie, 2017); pricing externalities to balance public the risks and benefits of scientific research generally (Farquhar, Cotton-Barratt, & Snyder-Beattie, 2017); and proposing a general international regulatory regime to govern global catastrophic and existential risks from emerging technologies (Wilson, 2013). At present, a majority of existential risk research centres3 have articulated law and policy research as areas of interest, and scholars in this space have begun to translate such work into concrete proposed policy interventions—notably the 2017 GPP report, which included proposals to develop governance for geoengineering research; establish international scenario plans and exercises for engineered pandemics, and build international attention for existential risk reduction (Farquhar, Halstead et al., 2017).¶ Such work is highly encouraging, and the existential risk research agenda has benefited from it. Nonetheless, the risk remains that a too-narrow conception of ‘existential risks’ prematurely closes down the space of law and governance solutions that are possible—or necessary—in assuring humanity a non-catastrophic future—for instance, a future that, in Bostrom’s framing, meaningfully ‘maximize[s] the probability of an ok outcome’ (Bostrom, 2013, p. 19). However, if human extinction and the persistent and pervasive truncation of technological potential are not completely homologous, then tailoring our portfolio of policy responses exclusively to closing off the pathways these risks could take—and then calling it a day—would be insufficient. In fact, this might only afford future policymakers with a false sense of security, even as the world continues to reside in an overall state of ‘super-risk’ (Bermudez & Pardo, 2015).¶ This is especially the case when there is a narrow ‘technological’ (re)solution on offer—such as ‘improve global vaccine synthesis and production capability’, or ‘subsidize international technical AI safety research’—which promise to address or prevent the risk at its root. While such direct technological solutions may certainly be indispensable to averting some existential risks, they may not suffice in actually ‘plugging all the holes’ in our risk space. In a disciplinary context, there is a risk (admittedly self-correcting, given publication incentives) of the research agenda ‘halting’ early. In a real-world context, the availability of simple, straightforward ‘fixes’ might even pose a ‘moral hazard’, if policymakers or global governance systems which lack political will or the attention to explore more complex or costly changes, seize upon the ‘symbolic action’ of the straightforward, first-order mitigation strategies. Even where this is not the case, certain policy recommendations to mitigate existential risks might depend on too-optimistic a view of institutional rationality or capability.¶ 1.2. ‘Boring Apocalypses’: from existential hazards, to existential risks¶ While such efforts might mitigate specific existential risks, this might not translate into significantly lowering the overall probability of the ‘adverse outcome’, if only a part of the problem, or only one problem among many, is addressed. An alternative articulation is that only one path to the ‘adverse outcome’ is being explored by much research into existential risks: erecting obstacles along that path may indeed reduce the overall likelihood of manifesting these risks, but this might have little impact, or even no effect, upon the manifestation of the ‘adverse outcome’.¶ Thus, our view is that a materialised existential risk (what we call an ‘existential hazard’) is sufficient to lead to an (existentially) ‘adverse outcome’, but crucially, that this is unnecessary to reach that result. If the overarching objective is to lower the probability of human extinction or significant technological curtailment, adopting an array of approaches which complement the mitigation of direct existential risks are required. Within this broad spectrum of aligned approaches, we propose to introduce law, policy, regulatory and governance tools in this paper as an example. The choice of law and policy perspectives is two-fold: on one hand, they make it possible to take second-order considerations, which involve indirect and socially and culturally mediated paths towards ‘adverse outcomes’ into account; on the other hand, these recognise both the complexity of social organisation and the prospect that civilizational collapse may trigger or possibly instantiate existential outcomes. In this sense, law and policy approaches offer the possibility of complementing and enhancing the narrower approach adopted by contemporary existential risk research, to take into consideration other paths to existentially adverse outcomes; and to better anticipate vulnerabilities, exposures and failure modes in societal efforts to address existential risks.¶ 1.3. Exploring the implications of the existential risk framing: risks from AI¶ An example of this can be drawn from the prospect of super intelligent artificial intelligence (Bostrom, 2014; Yudkowsky, 2008a). Although the landmark research agenda articulated by Russell, Dewey, and Tegmark (2015) does call for research into ‘short-term’ policy issues, debates in this field of AI risk4 have—with some exceptions—identified the core problem as one of value alignment, where the divergence between the interests of humanity and those of the superintelligence would lead to the demise of humanity through mere processes of optimisation. Thus, the existential risk posed by the superintelligence lies in the fact that it will be more capable than we can ever be; human beings will be outmanoeuvred in attempts at convincing, controlling or coercing that superintelligence to serve our interests. As a result of this framing, the research agenda on AI risk has put the emphasis on evaluating the technical feasibility of an ‘intelligence explosion’ (Chalmers, 2010; Good, 1964) through recursive self-improvement after reaching a critical threshold (Bostrom, 2014; Sotala, 2017a; Yudkowsky, 2008a, Yudkowsky, 2013)5; on formulating strategies to estimate timelines for the expected technological development of such ‘human-level’ or ‘general’ machine intelligence (Armstrong and Sotala, 2012, Armstrong and Sotala, 2015; Baum, Goertzel, & Goertzel, 2011; Brundage, 2015; Grace, Salvatier, Dafoe, Zhang, & Evans, 2017; Müller & Bostrom, 2016); and on formulating technical proposals to guarantee that a superintelligence’s goals or values will remain aligned with those of humanity—the so-called superintelligence ‘Control Problem’ (Armstrong, Sandberg, & Bostrom, 2012; Bostrom, 2012, Bostrom, 2014; Goertzel & Pitt, 2014; Yudkowsky, 2008a).6¶ While this is worthwhile and necessary to address the potential risks of advanced AI, this framing of existential risks focuses on the most direct and causally connected existential risk posed by AI systems. Yet while super-human intelligence might surely suffice to trigger an existential outcome, it is not necessary to it. Cynically, mere human level intelligence appears to be more than sufficient to pose an array of existential risks (Martin, 2006; Rees, 2004).¶ Furthermore, some applications of ‘narrow’ AI which might help in mitigating against some existential risks, might pose their own existential risks when combined with other technologies or trends, or might simply lower barriers against other varieties of existential risks. To give one example; the deployment of advanced AI-enhanced surveillance capabilities7—including automatic hacking, geospatial sensing, advanced data analysis capabilities, and autonomous drone deployment—may greatly strengthen global efforts to protect against ‘rogue’ actors engineering a pandemic (“preventing existential risk”). It may also offer very accurate targeting and repression information to a totalitarian regimes,8 particularly those with separate access to nanotechnological weapons (“creating a new existential risk”). Finally, the increased strategic transparency of such AI systems might disrupt existing nuclear deterrence stability, by rendering vulnerable previously ‘secure’ strategic assets (“lowering the threshold to existential risk”) (Hambling, 2016; Holmes, 2016; Lieber & Press, 2017).¶ Finally, many ‘non-catastrophic’ trends engendered by AI—whether geopolitical disruption, unemployment through automation; widespread automated cyberattacks, or computational propaganda—might resonate to instil a deep technological anxiety or regulatory distrust in global public. While these trends do not directly lead to catastrophe, they could well be understood as a meta-level existential threat, if they spur rushed and counter-productive regulation at the domestic level, or so degrade conditions for cooperation on the international level that they curtail our collective ability to address not just existential risks deriving from artificial intelligence, but those from other sources (e.g. synthetic biology and climate change), as well.¶ These brief examples sketch out the broader existential challenges latent within AI research and development at preceding stages or manifesting through different avenues than the signature risk posed by superintelligence. Thus, addressing the existential risk posed by superintelligence is both crucial to avoiding the ‘adverse outcome’, but simultaneously misses the mark in an important sense.¶ 2. Re-examining existential risks: hazard, vulnerability, and exposure¶ While Bostrom’s leading typology identifies the general area inhabited by existential risks, it provides little guidance for how to differentiate among the diverse risks within that category (the box marked ‘X’), because these risks are not distinguished according to their source, characteristics, or complexity, but only their impact (“crushing”) and scope (“pan-generational”).9¶ However, given the range of distinct risks falling within the ‘X’ box—that is, risks that could cause or feed into an eventual terminal and crushing ‘adverse outcome’ for humanity—we suggest it relevant to deconstruct existential risks, and instead consider the broader category of ‘risks as a function of hazard, vulnerability and exposure’10,11:¶ Existential Risk = Hazard* Vulnerability * Exposure¶ Here, hazard denotes the external source of peril (which is captured within the prevailing agenda studying existential risks)—the ‘spark’ that threatens the pan-generational/crushing harm.¶ Vulnerability denotes propensities or weaknesses inherent within human social, political, economic or legal systems, that increase the likelihood of humanity succumbing to pressures or challenges that threaten existential outcomes.¶ Finally, exposure denotes the ‘reaction surface’—the number, scope and nature of the interface between the hazard and the vulnerability.¶ Thus, a hazard is what kills us, and a vulnerability is how we die. Exposure is the interface or medium between what kills us, and how we die. To take an example from disaster studies, a major earthquake only becomes a risk if the built, social or institutional environment can be destabilised during earthquakes of the threatened magnitude (“is vulnerable to”), and if such an environment is located in (“exposed to”) an earthquake zone. Thus, vulnerability and exposure refer to two different aspects of the affected system: how it breaks, and how it intersects with a given hazard’s operating space or pathways of impact (Fig. 1).¶ As a species of global catastrophic risks, the study of existential risks is often conflated with, and perhaps even collapsed into, the identification and mitigation of existential hazards. Where attention is paid to issues of vulnerability and exposure, these are often identified in light of an existential hazard. One of the leading sources and reference points in the field symptomatically organizes the field as a collection of existential hazards (Bostrom & Cirkovic, 2008). A caveat applies for a small subset of hazards of such enormous magnitude that it renders mitigation strategies focussing upon vulnerability and exposure less relevant, or perhaps even irrelevant. The paragon might be the scenarios of ‘simulation collapse’, or a high-energy physics experiment going awry, altering the astronomical vicinity and rendering life untenable (Ord et al., 2010). Such extreme hazards constitute the archetype of existential risks as a subset of global catastrophic risks and can only be addressed by managing the hazard head-on, with vulnerability and exposure components relegated to marginal roles:¶ Existential Risk = Existential Hazard * Vulnerability * Exposure¶ Thus, our claim is not that the field of existential risks research is looking in the wrong places—the emphasis on existential risks has enabled this field to identify a core group of existential hazards which would on their own suffice to bring about the ‘existentially adverse outcome’.¶ Nonetheless, there are also many other, slower and more intertwined ways in which the world might collapse, without being hit by spectacular hazards. To complement the study of existential risks we can draw upon lessons learnt through historical and anthropological studies of civilizational collapse. Thus, while existential risks concentrate upon clear-cut existential hazards, civilizational collapse research infers influential factors that were involved in trajectories of decline. These studies are beginning to challenge the traditional conceptual framework which set out a cyclical history, wherein a civilisations rise and fall, progressing through a predictable pattern of growth, zenith and decline in a gradual manner (Ferguson, 2011). In other words, historically civilizational collapses are boring. Diamond refined this model by recognising that civilizational collapse could be a slow and protracted process emerging from complex interactions (Diamond, 2006).¶ 3. Beyond hazards: vulnerability and exposure¶ In this paper, we set out to foreground the other two variables involved in the existential risk equation.¶ Thus, as noted, ‘vulnerability’ denotes propensities or weaknesses inherent within human social, political, economic or legal systems that increase the likelihood of humanity succumbing to pressures or challenges that threaten existential outcomes.¶ ‘Exposure’ indicates the nexus between external hazards and internal vulnerabilities: the interface at which the ‘adverse outcome’ precipitates from their interaction. Historical studies of civilizational collapses indicate that even small exogenous shocks can destabilise a vulnerable system (Diamond, 2006; Ferguson, 2011). Given this, studying ‘exposure’ is relevant in systematically analysing interaction effects: a cataclysmic hazard interacting with robust and resilient human systems may be survivable, but conversely, at the interstices at which our human technology, institutions or culture are most vulnerable, even minor (initially ‘non-catastrophic’) hazards can be the inflection point that tips these susceptible systems towards trajectories of collapse (Gladwell, 2001).12¶ In order to offset the tight coupling between existential risks and existential hazards, we will further dissect the vulnerability and exposure factors introduced in the existential risk calculus. Our proposed taxonomy distinguishes four general categories of vulnerability and exposure (see Table 1).¶ •Ontological: vulnerability through existing in a given location and time in our universe13;¶ •Passive: vulnerability through lack of action; ‘indirect’ exposure;¶ •Active: vulnerability because of insufficient/mis-specified action.¶ •Intentional: vulnerability or exposure knowingly maintained, for that purpose.¶ Note that for vulnerability, the Passive, Active and Intentional categories correspond to the jurisprudential concepts of ‘omission’ (‘failure to act’), ‘negligence’ (action, but with failure to exercise the appropriate care to prevent foreseeable future harm) and ‘intention’ (action with the known purpose to bring about a consequence).¶ Drawing such distinctions offers the opportunity to be more precise about the features or characteristics which give rise to the existential dimension of the challenge, and thus suggest specific points for targeted intervention, as well as potential failure modes to caution against.¶ Below, we combine these categories and their sub-divisions, in twin taxonomies of existential ‘vulnerabilities’ and ‘exposure’. We also seek to give concrete examples. Obviously, not all of these examples are currently unstudied—indeed many feature prominently in the existing literature—though in other cases they remain understudied. While this list is naturally not comprehensive, we hope that such examples enable researchers in the field of existential risks to locate their research in an overarching framework, as well as facilitating links to established scholarly fields which have studied given issues, without considering their bearing on larger existential risks.¶ 3.1. A taxonomy of ‘existential vulnerability’¶ Our proposed taxonomy for distinguishing between different manifestations of existential vulnerabilities is summarised in Table 2: note that the salience or tractability of these existential vulnerabilities to law and policy approaches increases as one goes down: ontological vulnerabilities appear (at present) highly intractable to mere law and policy—it would be a vain regulator indeed who would try to legislate against physical laws. However, as one proceeds to passive, active, or intended vulnerabilities, the salience of governance approaches increases (Table 3).¶ 3.1.1. Ontological vulnerability¶ The category of ontological vulnerability denotes intrinsic vulnerabilities associated with human existence. These include the possibility that we inhabit a computer simulation (Bostrom, 2002), which might be terminated or altered at any time. More conceptual and basic vulnerabilities—so fundamental that we often would not even consider them as such—include our existence as biological beings that are dependent (potentially more so than other species such as tardigrades) on continuous or relatively uninterrupted inputs of energy & resources (such as food, water, air, light, …), which renders the human species one comparatively vulnerable to ‘extinction’ events such as a supervolcano- or meteor-induced global winter. On a deeper level yet, all biochemistry is dependent on the existing laws of physics within which it evolved, rendering us acutely and terminally vulnerable to any processes (e.g. vacuum decay) which would profoundly alter these processes. Biological deterioration due to aging processes, or exterior damages, might also rank amongst these, although that is conditional on whether or not there exists a physical ‘hard ceiling’ to how far medical senescence research might extend human lifespans and reduce other vulnerabilities.¶ As these are background conditions at the frontiers of epistemology, we are unlikely to be able to unveil more than a fraction of these vulnerabilities. Also, as inherent features of human existence we have limited abilities to act effectively in this category. Perhaps the most utility we can extract from delimiting ontological vulnerability is to restrict its reach: in other words to leave this as a residual class of vulnerabilities inherent in existence.¶ 3.1.2. Vulnerabilities, passive and active; built, institutional and cultural¶ Passive vulnerabilities are characterised by inaction: the susceptibility to existential outcomes by virtue of failure to take appropriate measures. Conversely, active vulnerabilities arise in association with human activities, as by-products or unintended consequences.¶ Three cross-cutting sub-distinctions can also be made for both passive and active vulnerabilities: built, cultural, and institutional.¶ Built vulnerabilities are characterised by our (passive) failure to put into place relevant solutions or defences to existential challenges, or by our (active) failure to repair or correct the extant vulnerabilities in the legacy infrastructures we deploy, or the path-dependent ways we deploy them—even if we have such solutions or repairs at our disposal. Such solutions can in fact include some interventions proposed by the existential risk research agenda, such as an asteroid defence programme or the ability to systematically monitor for supervolcano eruptions (Denkenberger & Blair, 2018); they also cover the active existential risks posed by the technologies which humanity has introduced, but which go unfixed—such as architectural deficiencies creating intractable cybersecurity vulnerabilities in universally used computing chips. Because of the technical nature of engineered vulnerabilities, some of these are perhaps closest to the existing (policy) research agenda of the existential risk community—and at present some may consider that law and policy tools have less of a role to play, other than to coordinate efforts aimed at addressing them.¶ In contrast, top-down vulnerabilities resulting from suboptimal direction and coordination are captured by our sub-category of institutional vulnerability. Here, the line between active and passive is admittedly thin, where recklessness can be the distinguishing feature. Active institutional vulnerability may be characterised by failure to coordinate to address a known risk, such as climate change, or cyclical global economic melt-down. Passive institutional vulnerability may then be understood as directional and coordination failures that limit the scope of knowledge related to existential risks—perhaps an implicit ‘unwillingness to know’, which translates in an unwillingness to fund blue-sky research into charting ‘unknown unknowns’ (Rumsfeld, 2002).¶ Cultural vulnerability encompass the bottom-up societal dimensions, reflecting how certain social practices may affect susceptibility to existential challenges. Active cultural vulnerabilities include customary practices that facilitate the spread of pathogens, increasing susceptibility to pandemics, for example integrated commercial travel networks and interpersonal greeting rituals which encourage physical proximity or contact. Passive cultural vulnerabilities include the exclusion or ridicule of existential risks from serious discussion in public forums (let alone the halls of power). This increases collective vulnerabilities insofar as the public and policymakers underrate the prospects for existential risks (cognitive biases exacerbate these effects, Kahneman, 2012) resulting in further marginalisation.¶ 3.1.3. Intended vulnerabilities¶ Intended vulnerabilities are those which are created or retained specifically for that purpose, and within the existing research agenda are reflected in the premises of the ‘AI risk’ or ‘Apocalyptic AI’ movement (Geraci, 2010). Another salient example can, however, be found in nuclear force postures which (in the US context) features centralization of launch command authority along with a ‘launch-on-warning’ doctrine that relies on input from fallible early launch warning systems (Borrie, 2014; Sagan, 1993). Together, this gives rise to the catastrophic risk of an accidental nuclear war (Barrett, Baum, & Hostetler, 2013). Yet far from incidental, this is arguably by design. As the theorist Kenneth Boulding once observed: “if [deterrence] were really stable … it would cease to deter. If the probability of nuclear weapons going off were zero, they would not deter anybody” (Boulding, 1986, p. 32).14 The nuclear force knowingly renders itself more vulnerable to catastrophic accidents—sacrificing a degree of safety for the sake of strengthening operational readiness and deterrence. While less dramatic, similar intentional vulnerabilities could emerge from a state intelligence service knowingly holding back-doors or ‘zero-day-exploits’ which it identifies in critical infrastructure software, in the hope that this may enable more effective cyberattacks against rival states at a later state.¶ 3.1.4. Existential vulnerability: mitigation and adaptation strategies¶ This taxonomy of vulnerabilities can provide concrete suggestions for addressing existential risks. While the categories of ontological and intended vulnerabilities may seem superfluous, their treatment as additional classes allow limited resources to be concentrated into the most tractable areas. Perhaps the main contribution of this taxonomy is to highlight how existential risks need not be active and discernible, in the manner of the ‘hazards’ identified in the field. Instead, many of these risks can be latent, and slow-moving. Moreover, this taxonomy aids in understanding how human activities can impact paths towards ‘existential outcomes’ in several ways: (1) intent: by directly creating technologies which pose existential hazards (i.e. emerging technologies such as AI, nanotechnology and synthetic biology); (2) (negligence) by establishing complex systems for which failure is unavoidable (Perrow, 2011); (3) and by omission, the failure to take steps to confront existential risks.¶ Beyond merely refining the sources of existential risks, the contribution of this taxonomy lies in creating a roadmap for the study and integration of risks that have not yet received much or consistent attention in the field of existential risks. In doing so, we emphasise a number of existential vulnerabilities, such as global dependency upon a few species of staple crops, or certain types of globalised technologies (e.g. SCADA-based systems in critical infrastructure) that are not commonly recognised as sources or failure points of existential risks.¶ The study of existential ‘vulnerability’ may suggest that adaptation strategies are preferable to those of mitigation, both because of the inherent complexity underlying both forms of structural vulnerability and because adaptation can now occur simultaneously with mitigation. This is because the vulnerability analysis in effect opens up a parallel system where other trajectories of existential risks are at play. The rough equivalence drawn between traditional existential risks with existential hazards might have the effect of underselling adaptation strategies: it is illogical to conceive of robustness as a defence against the apocalypse, after all. Along with efforts to mitigate or avert existential hazards, however, we can now also plan for adaptation against vulnerabilities. Thus, adaptation strategies are not limited to actions undertaken after ‘the Fall’: instead they may become rational reactions towards limiting susceptibility to existential risks. In order to explore this potential further, we proceed to examine a taxonomy of exposure.¶ 3.2. A taxonomy of ‘Existential exposure’¶ As a parallel effort to our taxonomy on existential vulnerabilities, we set out a classification system to differentiate between different forms of exposure. It is worth recalling at this point that we use exposure to express the interface between hazards and vulnerabilities—between what kills us, and how we die. Both hazards and vulnerabilities in isolation remain as potentials: exposure is thus a means of actualising such potential into existential risks.¶ Such exposure can further be directed towards either the societal or the natural environment. This is about what is directly at risk: our (human) society and the common capabilities and support structures preventing existential risks; or nature and its carrying capacity and resilience to future shocks. Thus, we assert that devastating results for humankind can follow from the collapse of both the societal structures we have built, as well as the natural environments within which these constructed systems are embedded. Again, the distinction allows us to single out different examples and trajectories to build alternative strategies for human survival. As is clear from the examples above, it also draws out lessons for existential outcomes which might not be immediately evident from an analysis of existential hazards alone. For example, when ‘exposure’ is seen from the perspective of the natural environment on which mankind depends, pervasive over-fishing and deforestation, combined with trends in resource demands tracking population growth, may become potentially hazardous activities with the potential to curtail human development in the long run (Diamond, 2006), even if they do not affect most humans directly in the short run.¶ 3.2.1. Ontological exposure¶ Some exposures are inherent in residing on Earth. Those falling in the category of natural exposure denote existence on earth itself as the exposure, and include our exposure to Near-Earth Objects (NEO) hitting earth or supervolcanoes triggering a protracted volcanic winter. The common denominator underlying this form of exposure is their requirement for measures beyond our present technological capacity to overcome (which admittedly, can be a moving threshold).¶ 3.2.2. Indirect and direct exposure¶ As with the discussion of existential vulnerabilities set out above, the potential of our proposed taxonomy lies in the analysis of indirect and direct exposures. This distinction identifies the exposures that are a direct consequence of human activity, from those that are caused by more complex interactions with other systems.¶ The theoretical example of high-energy physics research going awry15 provides an example of societal exposure.16 A final example of direct exposures are private or unilateral attempts to undertake ‘Active SETI’—alternately called METI (‘Messaging to Extra-Terrestrial Intelligence’) (Zaitsev, 2006)—which might expose the rest of mankind to catastrophic risk, should any future contacted alien species prove hostile and capable of interstellar-scale interdiction. These examples illustrate how surfaces of direct exposure (and ways to reduce it) might be overlooked when concentrating upon the hazard alone.¶ Beyond direct exposures, an array of arrangements which jeopardise the human societies that have become dependent on them. This category includes any activity or arrangement which might expose the world to extinction through cascading effects. The development of critical common global infrastructures such as the internet, energy markets, and cultural and scientific harmonization might be classified as exposures, rather than vulnerabilities because these reveal new interfaces between hazards and vulnerabilities. Thus collapse of common infrastructures would trigger cascades which jeopardise civilizational sophistication at the global level (Wright, 2006), the edifice upon which humanity’s long-term potential has been built. Similarly, developments like urbanisation, intensification of agriculture, and even increasing global inequality () appear to be factors that create fault lines and further drive exposures to existential vulnerabilities. Here the exposure perspective shows us that only by certain actions or inactions do risks actually materialise fully against civilization.¶ 3.2.3. Intentional exposure¶ Finally, some of these exposures appear to exist intentionally, or at least knowingly or recklessly. The city of New Orleans, Louisiana, provides a microcosm of how dysfunctional behaviour, seen from an existential risk perspective, might be driven by human incentives or rationales operating at different orders. The city is, in design and position, incredibly vulnerable to its natural environment—pinched in between the Mexican Gulf and Lake Pontchartrain and built on the banks of the Mississippi River. Accordingly, some have argued that the most reasonable strategy following hurricane Katrina would have been to abandon the city permanently (Richards, 2011). Instead, the affected populations were given incentives to return, with the US government investing billions in the reconstruction of the city, aware that even with improved defences, the city remains unsafe (Cutter et al., 2014).¶ Similarly, many populations worldwide, from Tehran and Kathmandu, to San Francisco and Port-au-Prince, persist in known disaster-prone zones, for (legitimate) reasons of culture, history, identity or economy. The purpose of these examples is not to warn the populations of these cities, nor to judge their decision to remain: rather the point is that individuals and societies often make decisions based upon entirely different rationales than a concern for survival. This is an insight that seems to scale to any level of government. In simpler terms, sometimes we choose exposure over safety because of competing considerations, and while this might be productive from a cultural heritage perspective, it remains problematic when seen through the lens of existential risks.¶ 4. Are existential hazards necessary for existential risks?¶ Having set out taxonomies for differentiating between factors which influence existential risks the question remains whether all components are necessary to bring about an ‘adverse outcome’. Our initial claim was that existential hazards could be sufficient existential risks, but that they were not necessary to pose such risks.¶ Returning to the civilization collapse literature cited above, Ferguson provides a critical insight in contesting the traditional view of cyclical history itself. He posits an alternative conceptual framework by asking the question: ‘What if history is not cyclical and slow-moving, but arrhythmic?’ (Ferguson, 2011, p. 299). Continuing, he summarises the perspective we adopt succinctly:¶ Civilisations… are highly complex systems, made up of a very large number of interacting components that are asymmetrically organised, so that their construction more closely resembles a Namibian termite mound than an Egyptian pyramid. They operate somewhere between order and disorder − on ‘the edge of chaos’, in the phrase of computer scientist Christopher Langton. Such systems can appear to operate quite stably for some time, apparently in equilibrium, in reality constantly adapting. But there comes a moment when they “go critical”. A slight perturbation can set off a “phase transition” from a benign equilibrium to a crisis − a single grain of sand causes an apparently stable sandcastle to fall in on itself. (Ferguson, 2011, pp. 299–300).¶ Wright echoes this sentiment: ‘Civilisations often fall quite suddenly − the House of Cards effect − because as they reach full demand on their ecologies, they become highly vulnerable to natural fluctuations’ (Wright, 2006, p. 130). When combined with the observation that hitherto isolated civilizational experiments have now been merged (Harari, 2015), this raises the spectre that existential risks can coalesce from factors that historically brought about only limited civilizational collapses. Thus, the question we need to pose in this regard is whether vulnerabilities themselves contain the seeds of existential risks?¶ In this context, we should note that vulnerabilities have often been considered mostly as aggravating factors. As aggravators then, vulnerabilities are subsidiary considerations restricted to influencing borderline events: where a potential existential hazard impacts humanity, its susceptibility or resilience could determine whether or not that hazard was transmuted into an existential outcome.¶ In line with vulnerabilities being developed as a separate sphere where existential risks are at play, this section explores the possibility of removing the existential character of the hazard and thus plausibly reducing the calculus to:¶ Existential Risk = Hazard * Existential Vulnerability * Exposure¶ [and/or]¶ Existential Risk = Hazard * Vulnerability * Existential Exposure¶ An initial issue is that a catalyst of some sort is required to precipitate the existential risk, because even a system with well-exposed inherent susceptibilities will need something to set it motion. Removing the existential hazard component allows us to explore the possibility that relatively minor occurrences can trigger cascades that emerge as existential risks. But a vulnerability cannot by definition transmute into the existential risk itself absent external input: for this reason we diminish the stature of ‘hazard’ in the equation to represent our proposition that exogenous shocks need not be the spectacular existential hazards recognised by the study of existential risks. Instead, the external hazards in our revised equation can include insignificant events which go unnoticed (and quite probably involve a large number of minor occurrences).¶ 5. Contributions and limitations of law and policy tools for existential risks¶ While our deconstruction of existential risks lead to fairly broad claims, it also provides a few concrete questions and insights. First and foremost, if existential risks can indeed be triggered by non-existential hazards, we need to broaden the scope of investigation in order to draw a more accurate roadmap of the existential risks field, one which can deal with questions of vulnerability and exposure explicitly.¶ Second, the type of perceived challenge channels the range of appropriate responses which can be developed. While existential hazards may appropriately be met by narrower forms of technical solutions and technologically-oriented mitigation strategies, our broader perspective of existential risks open up other toolboxes to confront existential risks. In particular, social vulnerability and human-driven (anthropogenic) exposure require improved governance and coordination for adaptation strategies. Thus, when we reconstruct existential hazards through the optics of the social systems’ inability to withstand them they, per definition, become social phenomena. As noted, many existential risk scholars have recently recognised the importance of reaching out to, and incorporating, law and governance approaches, even where the origin of the existential hazard itself is technological. The critical role of such law and governance approaches should be even more self-evident where the problems in question—the origins of existential vulnerability and exposure—are themselves social, not technological.