Search the archive.

(Daryl, 2024, Arms Control Association, “Nuclear Dangers and the 2024 Election”, https://www.armscontrol.org/act/2024-01/focus/nuclear-dangers-2024-election)

As the new year begins, the existential risks posed by nuclear weapons continue to grow. A crucial factor in whether one or more of today’s nuclear challenges erupt into full-scale crisis, unravel the nonproliferation system, or worse will be the outcome of the U.S. presidential election.

(EIU, 3-19-24, “Why Biden will win re-election in 2024”, https://www.eiu.com/n/why-biden-will-win-re-election-in-2024/)

Mr Biden and Mr Trump have now won enough delegates to secure their party nominations, aligning with our view that November’s presidential election will be a rematch of the 2020 race. Although Mr Biden trails Mr Trump in most current polls, we maintain our forecast that Mr Biden will win the election, despite the challenges that he faces. This article outlines the reasoning behind our call and the factors that could persuade us to change it.

AEs constitute a threat to humanity because the only limits on their conduct are the limits the least restrictive human creator imposes. As the science advances, algorithms’ abilities will improve until they far exceed those of humans. What remains to be determined is whether humans will be successful in imposing controls before the opportunity to do so has passed.

This Article has addressed a previously unexplored aspect of the artificial-intelligence-control problem. Giving algorithms control of entities endows algorithms with legal rights and gives them the ability to transact anonymously with humans. Once granted, those rights and abilities would be difficult to revoke. Under current law, algorithms could inhabit entities of most types and nationalities. They could move from one type or nationality to another, thereby changing their governing law. They could easily hide from regulators in a system where the controllers of nonpublicly-traded entities are all invisible. Because the revocation of AEs’ rights and abilities would require the amendment of thousands of entity laws, the entity system is less likely to function as a means of controlling artificial intelligence than as a means by which artificial intelligence will control humans.

Building on theories of phenomenology, post-Cartesian approaches to cognitive science, and critical studies, we ground our position in the lived reality of actual humans in an increasingly ubiquitously connected, controlled and surveilled society. What we find is the seamless integration of machinic systems into daily lives in the name of convenience and efficiency. The last thing these systems need is legally enforceable “rights” to ensure persons defer to them. Rights are exceptionally powerful legal constructs that, when improvidently granted, short-circuit exactly the type of democratic debate and empirical research that is necessary in our increasingly technologized world [27]. Conversely, the ‘autonomous intelligent machine’ is a sci-fi fantasy, a meme that functions to mask the environmental costs and human labour which form the backbone of contemporary AI. The robot rights debate further mystifies and obscures these problems. And it could easily provide a normative rationale for permitting powerful entities developing and selling AI to be absolved from accountability and responsibility.

Artificial Intelligence. An artificial intelligence (AI) program may broadly be defined as a computer program that automatically finds ways to achieve goals. For example, an AI chess program finds moves for achieving the goal of checkmating the opponent. The moves have not been explicitly defined by the programmer – instead the program finds them by “weighing” the pros and cons of different strategies. An AI question-answering system automatically generates the answer to a query by searching a knowledge base. A self-driving car finds a way to control throttle, break, and steering, in order to get to its goal destination.

An important distinction between different AI systems is generality. Most present-day AIs can only solve narrow sets of tasks in restricted environments. For example, most chess programs can only play chess, and most question answering systems can only answer specific type of queries from a particular kind of database. This is in contrast to humans, who can learn to perform a wide range of cognitive tasks.

2. The dangers of unfriendly powerful artificial general intelligence

Unlike with whatever wetware runs the human brain, it would be relatively easy to make changes to a PAGI’s software. PAGI could even make changes to itself. Such self-modification could possibly allow PAGI to undergo an intelligence explosion where it figures out how to improve its own intelligence, then, as it gets smarter, it figures out new ways to improve its intelligence. It has been theorized that through recursive self-improvement a PAGI could go from being a bit smarter than humans to becoming a computer superintelligence in a matter of days (Good, 1965; Yudkowsky, 2008).

DeepMind founder Demis Hassabis, in a recent interview about the promise and perils of AI, offered a note of caution. “I think a lot of times, especially in Silicon Valley, there’s this sort of hacker mentality of like ‘We’ll just hack it and put it out there and then see what happens.’ And I think that’s exactly the wrong approach for technologies as impactful and potentially powerful as AI. … I think it’s going to be the most beneficial thing ever to humanity, things like curing diseases, helping with climate, all of this stuff. But it’s a dual-use technology — it depends on how, as a society, we decide to deploy it — and what we use it for.”

Other leading AI labs are simply skeptical of the idea that there’s anything to worry about at all. Yann LeCun, the head of Facebook/Meta’s AI team, recently published a paper describing his preferred approach to building machines that can “reason and plan” and “learn as efficiently as humans and animals.” He has argued in Scientific American that Turing, Good, and Hawking’s concerns are no real worry: “Why would a sentient AI want to take over the world? It wouldn’t.”

Building on theories of phenomenology, post-Cartesian approaches to cognitive science, and critical studies, we ground our position in the lived reality of actual humans in an increasingly ubiquitously connected, controlled and surveilled society. What we find is the seamless integration of machinic systems into daily lives in the name of convenience and efficiency. The last thing these systems need is legally enforceable “rights” to ensure persons defer to them. Rights are exceptionally powerful legal constructs that, when improvidently granted, short-circuit exactly the type of democratic debate and empirical research that is necessary in our increasingly technologized world [27]. Conversely, the ‘autonomous intelligent machine’ is a sci-fi fantasy, a meme that functions to mask the environmental costs and human labour which form the backbone of contemporary AI. The robot rights debate further mystifies and obscures these problems. And it could easily provide a normative rationale for permitting powerful entities developing and selling AI to be absolved from accountability and responsibility.

These concerns are not as pressing if AGI is impossible, requires a paradigm shift away from deep learning, or some fundamental breakthrough in our understanding of intelligence. However, we may not be so lucky. There is a growing opinion that we may be able to achieve AGI with techniques similar to today’s deep learning systems with ever more compute. This scenario is called “prosaic AGI,” where we achieve AGI through prosaic (mundane) techniques. This possibility has begun to be taken seriously due to recent advances in large NLP language models pioneered by the transformers architecture. Computer scientists have observed that the training loss for models can decrease linearly with respect to additional compute [8]. As compute becomes cheaper as we follow Moore’s Law, we may observe a trend where existing models become more powerful without significant technological advances. Progress may accelerate by coupling prosaic advances with genuine computer science breakthroughs.

The key takeaway is that if this prosaic approach provides a plausible path to AGI, we may have less time to address many problems with deep learning that can pose risks to humans. Contemporary problems around discriminatory algorithms may become even more damaging as these models become more powerful and effective. A virtuous tool for good actors that can quickly become a weapon in the hands of bad actors. Furthermore, as we’ll explore in the next section, there are many open questions about how we’ll align AGI systems with human values.

Yet another serious threat comes from artificial intelligence, or AI. In the near-term, AI systems like those sold by IBM, Microsoft, Amazon and other tech giants could exacerbate inequality due to gender and racial biases. According to a paper co-authored by Timnit Gebru, the former Google employee who was fired "after criticizing its approach to minority hiring and the biases built into today's artificial intelligence systems," facial recognition software is "less accurate at identifying women and people of color, which means its use can end up discriminating against them." These are very real problems affecting large groups of people that require urgent attention.

But there are also longer-term risks, as well, arising from the possibility of algorithms that exceed human levels of general intelligence. An artificial superintelligence, or ASI, would by definition be smarter than any possible human being in every cognitive domain of interest, such as abstract reasoning, working memory, processing speed and so on. Although there is no obvious leap from current "deep-learning" algorithms to ASI, there is a good case to make that the creation of an ASI is not a matter of if but when: Sooner or later, scientists will figure out how to build an ASI, or figure out how to build an AI system that can build an ASI, perhaps by modifying its own code.

A recent Washington Post article about artificial intelligence (AI) briefly caught the publics’ attention. A former engineer working for Google’s Responsible AI organization went public with his belief that the company’s chatbot was sentient. It should be stated bluntly: this AI is not a conscious entity. It is a large language model trained indiscriminately from Internet text that uses statistical patterns to predict the most probable sequence of words. While the tone of the Washington Post piece conjured all the usual Hollywood tropes related to humanity’s fear of sentient technology (e.g., storylines from Ex Machina, Terminator, or 2001: A Space Odyssey), it also inadvertently highlighted an uncomfortable truth: As AI capabilities continue to improve, they will become increasingly effective tools for manipulating and fooling humans. And while we need not fear an imminent cyborg apocalypse, we do need to prepare and strategize for a new era of AI-enabled disinformation.

Disinformation operations — covert efforts to deliberately spread false or misleading information — have historically been a distinctively human endeavor. With the rise of instantaneous digital communications, malign actors have increasingly exploited the machine learning systems embedded in our daily lives to precisely target audiences, shape global public opinion, and sow social discord. Today, disinformation operators are expanding their manipulation toolkits to include new AI techniques. AI-generated synthetic media and convincing AI-enhanced chatbots now offer threat actors a growing array of persuasive, tailored, and difficult-to-detect messaging capabilities. While machine learning techniques can also be used to combat disinformation, they will likely remain insufficient to counterbalance the expanding universe of anonymous digital mercenaries. Unless liberal democracies develop whole-of-society counter-disinformation strategies, AI-enhanced disinformation operations will further exacerbate political polarization, erode citizen trust in societal institutions, and blur the lines between truth and lies.

More troubling outcomes are likely if advocates for humanoid robots claim human rights for them or program them to imitate a desire for such rights. Consider, for instance, the right to communicate online. Lawyer John Frank Weaver [74] has argued that “all the constitutional speech protections that humans enjoy in the United States” should “apply to A.I., robots, and Twitterbots, whether they are Russian intruders or Microsoft mistakes.” The proliferation of bots has already overwhelmed and drowned out actual human speech in several contexts, spamming hashtags and otherwise impeding communication and collective will formation. The larger purposes of free expression are to enable human autonomy, and to allow humans to democratically self-govern. These purposes are not served by expansions of free expression rights to bots (often built by already-powerful entities) that would permit them to dominate communicative spheres.

The first of these causes is disease, and specifically those diseases that are caused by microbes—loosely defined to include both bacteria and viruses. Inherited diseases can have appalling consequences for individuals, but they’re not a threat to our species. Diseases caused by animals, such as schistosomiasis, the disease agent here being a parasitic worm, can also have very adverse effects on the people suffering from them, but they are unlikely to cause human extinction. In contrast, diseases caused by microbes are a potential threat to the continued survival of humans on planet Earth.

The reasons for this assertion are as follows. The shorter the life cycle of an organism, the faster it can evolve. And when a disease agent has a very short life cycle and its host a very long one, the discrepancy between the two is most likely to result in an evolutionary advantage to the former. Also, the changing fates of host species in terms of their population numbers will be a major factor driving the evolution of the disease agent. Once upon a time, a few million years ago, the population numbers of protochimpanzees and protohumans were probably about equal. Now there are less than half a million chimps and more than 7 billion humans—so there are about 10,000 humans for every one chimp. If you were a bacterium currently specializing in chimps, what evolutionary trajectory would you choose?

But Deep Blue can only beat humans at chess. Artificial General Intelligence (AGI), by contrast, refers to any machine that can match humans across the board in a wide range of intellectual tasks. General intelligence is a far harder problem to solve, and progress here has been less dramatic than experts predicted it would be 50 years ago. In the last five years, however, breakthroughs in new visual and speech recognition systems have surprised AI researchers, and the most promising sign is re- search led by Demis Hassabis, the founder of Google DeepMind. The team at DeepMind re- ported developing a system that taught itself how to play 49 different Atari arcade games, such as Space Invaders, learning not only the rules but how to maximize its score. The re- searchers programmed the artificial agent with no background knowledge about the games, beyond telling it to score as high as it could. After playing the games many times, the AI worked out the best strategies for itself, includ- ing some the designers themselves were un- aware of. Hassabis describes this as “the first significant rung of the ladder toward proving a general learning system can work.” Of course, this rung is hardly threatening, except to de- signers of video games.

The most serious AI safety concerns arise not from narrow intelligence or even general intelligence, but rather from what many ex- perts argue will quickly follow AGI: artificial superintelligence. Because a capacity to learn would be an integral component of any system that attains general intelligence, the AI would be able to boost its own capabilities—such an AI would be able to design an improved version of itself, and then that smarter version would do the same, and so on. This recursive self-im- provement could result in an “intelligence ex- plosion,” a scenario that the mathematician I.J. Good, one of Alan Turing’s colleagues, first identified in 1965. In this scenario, a system’s intelligence increases, potentially even from a sub-human level in most areas but with specific capabilities in fields such as mathematics and computer science, to a lev- el far beyond that of humans in a relatively short period of time.

Even if Russia and the West avoid direct conflict this time, they might not be so lucky the next. As relations worsen, future disruption of critical Western infrastructure by Russian intelligence, such as the NotPetya and Olympic Destroyer attacks, are less likely to be viewed as mere crimes. Repeated crises bordering on war may further erode the tacit agreements and relative restraint of quieter times. After repeated iterations of intensifying cyber operations, both Russia and the West may feel their backs to the wall with few options left other than military force when the next crisis — physical or cyber — emerges. Under extreme conditions, some of the same characteristics that lead cyber capabilities to be a pressure release might have the opposite effect, a mechanism that Bob Jervis and I have described as the Escalation Inversion.

If Putin believes a direct conflict with NATO is likely and expects its adversaries to take measures to reduce vulnerabilities, he could conclude that the best possibility for success is to launch a massive preemptive cyber attack. Since the U.S. military may seem otherwise unbeatable, this may lead Russia to “compensate with audacity in order to redress the balance.” The more the United States brags about its overwhelming offensive cyber advantage, but frets over weak defenses, the more any adversary might feel the need to target the United States as early and as hard as possible.

The AIC literature reveals that, within the cybersecurity sphere, AI is taking on a malevolent and offensive role—in tandem with defensive AI systems being developed and deployed to enhance their resilience (in enduring attacks) and robustness (in averting attacks), and to counter threats as they emerge (Yang et al. 2018; Taddeo and Floridi 2018a). The 2016 DARPA Cyber Grand Challenge was a tipping point for demonstrating the effectiveness of a combined offensive–defensive AI approach, with seven AI systems shown to be capable of identifying and patching their own vulnerabilities, while also probing and exploiting those of competing systems. More recently, IBM launched Cognitive SOC (“Cognitive Security—Watson for Cyber Security | IBM” 2018). This is an application of a machine learning algorithm that uses an organisation’s structured and unstructured security data, including content extracted from blogs, articles, reports, to elaborate information about security topics and threats, with the goal of improving threat identification, mitigation, and responses. Of course, while policies will obviously play a key role in mitigating and remedying the risks of dual-uses after deployment (for example, by defining oversight mechanisms), it is at the design stage that these risks are most properly addressed. Yet, contrary to a recent report on malicious AI (Brundage et al. 2018, 65), which suggests that “one of our best hopes to defend against automated hacking is also via AI”, the AIC analysis suggests that over-reliance on AI can be counter-productive. All of which emphasises the need for further research into AI in cybersecurity—but also into alternatives to AI, such as focussing on people and social factors.

Persons

In two recent articles, Professor Shawn Bayern demonstrated that anyone can confer legal personhood on an autonomous computer algorithm merely by putting it in control of a limited liability company (LLC).11 The algorithm can exercise the rights of the entity, making them effectively rights of the algorithm.

The rights of such an algorithmic entity (AE) would include the rights to privacy,12 to own property, to enter into contracts, to be represented by counsel, to be free from unreasonable search and seizure,13 to equal protection of the laws,14 to speak freely, and to spend money on political campaigns.15 Once an algorithm had such rights, Bayern observed, it would also have the power to confer equivalent rights on other algorithms by forming additional entities and putting those algorithms in control of them.16

According to Yampolskiy and Spellchecker (2016), the probability and seriousness of AI failures will increase with time. We estimate that they will reach their peak between the appearance of the frst self-improving AI and the moment that an AI or group of AIs reach global power, and will later diminish, as late-stage AI halting seems to be a low-probability event.

AI is an extremely powerful and completely unpredictable technology, millions of times more powerful than nuclear weapons. Its existence could create multiple individual global risks, most of which we can not currently imagine. We present several dozen separate global risk scenarios connected with AI in this article, but it is likely that some of the most serious are not included. The sheer number of possible failure modes suggests that there are more to come.

Even so, any sort of legal personhood for AIs would be a dramatic legal change that could prove problematic.225 As discussed earlier, providing legal personality to AI could result in increased anthropomorphisms. People anthropomorphizing AI expect it to adhere to social norms and have higher expectations regarding AI capabilities.226 This is problematic where such expectations are inaccurate and the AI is operating in a position of trust. Especially for vulnerable users, such anthropomorphisms could result in “cognitive and psychological damages to manipulability and reduced quality of life.”227 These outcomes may be more likely if AI were held accountable by the state in ways normally reserved for human members of society. Strengthening questionable anthropomorphic tendencies regarding AI could also lead to more violent or destructive behavior directed at AI, such as vandalism or attacks.228 Further, punishing AI could also affect human well-being in less direct ways, such as by producing anxiety about one’s own status within society due to the perception that AIs are given a legal status on a par with human beings.

Finally, and perhaps most worryingly, conferring legal personality on AI may lead to rights creep, or the tendency for an increasing number of rights to arise over time.229 Even if AIs are given few or no rights initially when they are first granted legal personhood, they may gradually acquire rights as time progresses. Granting legal personhood to AI may thus be an important step down a slippery slope. In a 1933 Supreme Court opinion, for instance, Justice Brandeis warned about rights creep, and argued that granting corporations an excess of rights could allow them to dominate the State.230 Eighty years after that decision, Justice Brandeis’ concerns were prescient in light of recent Supreme Court jurisprudence such as Citizens United v. Federal Election Commission and Burwell v. Hobby Lobby Stores, which significantly expanded the rights extended to corporations.231 Such rights, for corporations and AI, can restrict valuable human activities and freedoms.

While a lot of criticism has been made against possibility of AI confinement, a lot of it is based on Yudkowsky’s AI-Boxing experiments. Careful analysis of the protocol used by Yudkowsky in conducting his AI-Box experiments reveals that they were unscientific and explicitly designed to show impossibility of confinement. Without getting into technical details, first, the participant selection was not random. In fact, Yudkowsky has handpicked participants among those who have participated in the singularity related forums run by Yudkowsky and so are disproportionately likely to subscribe to his world view. Second, participants have not undergone any AI-safety training as would be expected of any true guards tasked with supervising an AI-Box. Also, the guards were given a right to release AI at their own discretion without a chance to consult with anyone and without any punishment for failing to keep AI contained; all are highly unrealistic conditions for a supposed prison security guard. The communication itself was completely open with no filters of any kind and no 3rd party supervision. Also, the guards were required to communicate with the AI for a prolonged period of time. Finally, any valuable scientific insights from the experiments were completely lost as one of the conditions of the protocol stated that ‘... neither party shall ever reveal anything of what goes on within the AI-Box experiment...’ (Yudkowsky, 2002).

3.1 Countermeasures Against Escape