Defense doesn’t assume cascading and uncontrollable factors that are impossible to predict.

The ability of a totalitarian regime to endure over a long time is a critical determinant in assessing the threat it could pose. Authoritarian regimes tend to last longer than totalitarian ones and may arguably have been one of the most common forms of governance throughout human history. Caplan, writing about the durability of totalitarianism, notes that “the best thing one can say about totalitarian regimes is that the main ones did not last very long” (Caplan 2008, 507). It is unclear whether the short lifespan of history’s totalitarian regimes has been related to events of those times or if it is an inherent feature of totalitarian systems.

Technology plays a complex role in governance, with the overall effect of creating the possibility to enhance the power of those who control it. Bertrand Russell wrote in Authority and the Individual that technological development enables the possibility of more intense government control (Russell 1949). Technology and power are inexorably interconnected, and the control offered by technological power was essential to the rise of totalitarianism in the twentieth century (van der Laan 1997). Many tensions exist in the relationships between totalitarianism and technology. Technological development is complex and unwise to halt. The profits to be made, improvements to the quality of life, and the ability to mitigate disaster ensure that advancements in most emerging technologies will continue. At the same time, these technologies may enable totalitarian regimes to rise and stay in power. Further, the ability to mitigate many existential risks discussed in this book, such as an engineered pandemic, may require building sophisticated surveillance systems, which, in turn, could be the very technology that allows a totalitarian regime to take control. It may be difficult to garner political will behind the risk of not being able to cure most diseases or increase lifespan for fear of being killed by a possible future regime in the next century (Caplan 2008).

It is possible that the short life of these regimes was due in part to a lack of sufficiently powerful technologies. If totalitarianism aims to control human nature, there are several possibilities for how emerging technologies could grant this power. Caplan highlights how emerging technologies used for control affect the likelihood of a global totalitarian regime coming to power. He argues that developing cheap and effective genetic screening for personality traits could shift the balance of probability. Caplan argues that if this technology comes to pass and humanity maintains reproductive control at the individual level, it would decrease the 5% unconditional probability to 3%, while that same technology in the hands of a government could use it to favor traits amendable to being controlled and raise the probability of totalitarian takeover to 10% (Caplan 2008). Following this line of thought, Caplan argues it can be inferred that if Stalin and Hitler had access to advanced digital technology and biotechnologies, their regimes might have endured much longer.

However, a totalitarian regime could gain and maintain total control without using technology to alter the genes of its population. Advanced smart weapons could be used in lieu of, or in conjunction with, secret police forces. While individual officers may refuse to carry out atrocious acts, an automated system would have no moral objections (Tegmark 2018). Yet unimagined weapons developed through the nexus of AI, nanotechnology, and synthetic biology could lead to a tool for a hegemonic regime to be able to kill their opponents with weapons they did not even know existed (Tegmark 2018).

Propaganda is an essential tool for a totalitarian regime. Through mass media, a regime can influence its population both through the narrative they disseminate and by stating the one acceptable version of the truth, so that even those who disagree with what is being presented know what they are not allowed to say or think (Cassinelli 1960; Kecskemeti 1950; Arendt 1951). Media control has often been shown to make people less critical of political systems (Stockmann and Gallagher 2011).

Digital propaganda and disinformation techniques are rapidly evolving driven by increased computing power, lowered costs, and proof of effectiveness. These tools are no longer used primarily by one state against another, but are deployed domestically by states, nonstate actors, and criminals. One such tool is deepfakes, where Generative Adversarial Networks (GANs) are used to create hyper-realistic synthetic media. Fake audio and video can then be quickly spread through social networks. When applied at a strategic moment, deepfakes have already proven to be destabilizing to governments (Honigberg 2022).

In addition to their effects on governments, deepfakes can slow down the real news as journalists have to determine real sources from the synthetic. While there are currently effective approaches for analyzing and evaluating the validity of deepfakes, it takes time. Often a significant amount of damage has already been done before a deepfake can be identified as such. This technology’s larger and more haunting effect is that it diminishes people’s trust in authorities, media, and a sense of reality. Further, deepfakes provide a convenient method for denying real information. Mistrust of the media can be a powerful tool for totalitarianism in its own right, further perpetuating the “liar’s dividend,” allowing leaders to maintain support in the face of damaging information (Chesney and Citron 2019). Future advances in AI will increase the quality of deepfakes and other synthetic media, further exacerbating the divide between the real and the fake and will likely become a powerful disruptive tool if AI developments for use in deepfakes progress faster than the technology and methods used to detect them (Kietzmann et al. 2020; Anand and Bianco 2021).

Surveillance is another essential tool of the totalitarian regime. To control the human nature of a population, those in control need to know what the people are doing. To stay in power, possible dissent and rebellion must be monitored. Technology increases the research, speed, and effectiveness of surveillance programs. Emerging technologies may continue this trend to a level previously only imagined in science fiction movies. Max Tegmark (2018) describes a possible scenario:

With superhuman technology, the step from the perfect surveillance state to the perfect police state would be minute. For example, with the excuse of fighting crime and terrorism and rescuing people suffering medical emergencies, everybody could be required to wear a security bracelet that combined the functionality of an Apple Watch with continuous uploading of position, health status, and conversations overheard. Unauthorized attempts to remove or disable it would cause it to inject a lethal toxin into the forearm. (176)

Powerful surveillance systems already exist. One of the many revelations made public by Edward Snowden was that some governments were working to develop “full-take” digital surveillance systems. One of these programs is TEMPORA, a system used by the British Government Communications Headquarters that buffered internet traffic through the fiber-optic cables that make up the backbone of the internet. This information was taken by intelligence agencies in its entirety to be processed later (MacAskill et al. 2013). The aim of such programs may be to develop AI-augmented systems to interpret what these large amounts of data mean in real-time. These advancements are driven by the fact that, generally, computing power improves over time while getting cheaper, making large-scale networking and data mining increasingly feasible. Never before in human history have these technologies been as powerful as they are now, and there is no foreseeable reason why these technologies would not continue to advance (Phoenix and Treder 2008).

It is clear how powerful surveillance systems could empower totalitarian regimes. Similar powerful surveillance systems may, at the same time, be required to mitigate some existential risks. If, for example, future developments in synthetic biology make it frighteningly easy to develop a dangerous pathogen with little skill and resources, this would be the equivalent of discovering that a nuclear bomb could be made by combining common household ingredients in the microwave. Even if the vast majority of people would never consider using this newfound power, it would only take a few individuals to destabilize the world. In such a scenario, governments would have strong incentives to develop real-time global surveillance systems to know what every individual is doing at every moment. Such a system is not impossible, though currently estimated to cost about 1% of global Gross Domestic Product (Bostrom 2019). This system, though developed to safeguard against an existential threat, would be an avenue for the rise of a totalitarian power.

Other emerging technologies initially intended for different uses could also empower totalitarian regimes. Research has already demonstrated the feasibility of using machine learning as a “lie detector” whereby videos of courtroom trials are used to train a program to read micro-expressions and other indicators of deception (Wu et al. 2017). Advanced forms of this technology, if developed initially with questionably benign intent for use in law enforcement, could easily co-opted into a powerful tool for human control, what George Orwell described as “how to discover against his will, what another human being is thinking” (1982, 399). Interest in developing this type of technology already exists. Barring regulation, it is likely that increasingly advanced systems for “knowing what someone is thinking” will be developed (Bittle 2020; Gaggioli 2018). Such a tool would greatly benefit a totalitarian regime that wanted to remake human nature.

One weakness of totalitarian governments has been the problem of succession. When the party leader dies, a power struggle for the leader’s successor typically ensues. Often the leader who follows does not hold as stringently to the same system set up by the founder. Nikita Khrushchev followed Joseph Stalin and was later peacefully removed as new leadership came in that was less dogmatic in its adherence to ideology, eventually resulting in Mikhail Gorbachev and his reforms. Then the Soviet Union collapsed in 1991. A similar process occurred in China when Deng Xiaoping succeeded Mao Zedong after his death and reduced the emphasis on Maoist ideology leading to the move toward more free markets (Caplan 2008).

Advancements in biotechnology have already been directed at life-extension and antiaging research (Parrish 2019; Stambler 2019; Abramson 2019). This technology initially developed without the intentions of political control in mind could, in effect, solve the succession problem by ensuring the long or indefinite life of leaders, possibly making their regimes more enduring (Caplan 2008). Additionally, advancements in genetic engineering and behavioral genetics could be used by a totalitarian regime to protect itself from dissent. Some strong components of political orientation may be genetic (Pinker 2002; Dawes and Weinschenk 2020; Hatemi et al. 2014). A sufficiently powerful regime could take these technologies away from their intended uses of treating inherited genetic conditions and use them for screening for dissent or possibly engineering compliance.

The nature of total control defined by totalitarian regimes is also one of their considerable weaknesses in dealing with catastrophes. A rigid governance structure that leads to a single decision-making point can be inflexible and fail to anticipate catastrophe. One reason for difficulty is the tendency in such regimes for aides and advisors to not share negative information with the leadership for fear it would be seen as dissent. Leaders, in this case, would have large information blind spots increasing their vulnerability to risks. When idiosyncratic aspects of the leader’s personality inform decision-making and top-level leadership if driven by individual motives or emotions rather than reason, this problem is amplified. These leaders may fail to anticipate and react to catastrophic threats even if they see them coming (Caplan 2008). The centralized control structure of these regimes is likely to lead to society reaching a static equilibrium with little continued technological advancement outside of those technologies of control. On a long enough timeline, this ensures human extinction as a result of any number of natural or manmade [anthropogenic] disasters is ensured (Cotton-Barratt and Ord 2015).

Totalitarianism increases existential risks indirectly through increasing societal and global vulnerability and decreasing resiliency. A totalitarian regime, like Pol Pot, Hitler, Stalin, or the Kim family, with access to previously described technologies would contribute to geopolitical instability and likely could be a significant factor in a Great Power Conflict.