RWG – AI Module Updates
We’re developing AI now and have passed the point of no return—extinction
Sean Martin, 11-3-2017, "Humanity’s days are NUMBERED and AI will cause mass extinction, warns Stephen Hawking," Express.co.uk, https://www.express.co.uk/news/science/875084/Stephen-Hawking-AI-destroy-humanity-end-of-the-world-artificial-intelligence
Professor Hawking has once again reiterated his claims manhumankind will inevitably fail, and says that our time on Earth is now numbered after we passed the point of “no return”. The theoretical physicist says that developments in AI have been so great that the machines will one day be more dominant than human beings. He told Wired Magazine: "I fear that AI may replace humans altogether. If people design computer viruses, someone will design AI that improves and replicates itself. “This will be a new form of life that outperforms humans.”
It's likely- arms racing makes mis development far more likely
Brian Tomasik 13 {Tech advisor for the Foundational Research Institute. 12-5-2013. “International Cooperation vs. AI Arms Race.” https://foundational-research.org/international-cooperation-vs-ai-arms-race/#AI_arms_races}//JM
AI arms races Government AI development could go wrong in several ways. Plausibly governments would botch the process by not realizing the risks at hand. It's also possible that governments would use the AI and robots for totalitarian purposes. It seems that both of these bad scenarios would be exacerbated by international conflict. Greater hostility means countries are more inclined to use AI as a weapon. Indeed, whoever builds the first AI can take over the world, which makes building AI the ultimate arms race. A USA-China race is one reasonable possibility. Arms races encourage risk-taking -- being willing to skimp on safety measures to improve your odds of winning ("Racing to the Precipice"). In addition, the weaponization of AI could lead to worse expected outcomes in general. CEV seems to have less hope of success in a Cold War scenario. ("What? You want to include the evil Chinese in your CEV??") With a pure CEV, presumably it would eventually count Chinese values even if it started with just Americans, because people would become more enlightened during the process. However, when we imagine more crude democratic decision outcomes, this becomes less likely. In Superintelligence: Paths, Dangers, Strategies (Ch. 14), Nick Bostrom proposes that another reason AI arms races would crimp AI safety is that competing teams wouldn't be able to share insights about AI control. What Bostrom doesn't mention is that competing teams also wouldn't share insights about AI capability. So even if less inter-team information sharing reduces safety, it also reduces speed, and the net effect isn't clear to me. Of course, there are situations where arms-race dynamics can be desirable. In the original prisoner's dilemma, the police benefit if the prisoners defect. Defection on a tragedy of the commons by companies is the heart of perfect competition's efficiency. It also underlies competition among countries to improve quality of life for citizens. Arms races generally speed up innovation, which can be good if the innovation being produced is both salutary and not risky. This is not the case for general AI. Nor is it the case for other "races to the bottom".
Risks of Astronomical suffering outweigh all other impacts, you should overcorrect against the optimism bias and the Disjunction fallacy
Althaus 16 (Reducing Risks of Astronomical Suffering: A Neglected Priority by and II.I Psychological factors First published: Sep. 2016. Last edited: August 2019. )
It is human nature to (subconsciously) flinch away from contemplating horrific realities and possibilities; the world almost certainly contains more misery than most or . Our tendency to underestimate the expected amount of future (as compared to present-day) suffering might be even more pronounced. While it would be unfair to apply this characterization to all people who display great optimism towards the future, these considerations certainly play a large role in the epistemic processes of some future “optimists.” One contributing factor is optimism bias (e.g. Sharot, Riccardi, Raio, & Phelps, 2007), which refers to the tendency to overestimate the likelihood of positive future events while underestimating the probability and severity of negative events – even in the absence of evidence to support such expectations. Another, related factor is , where people are prone to judging scenarios which are in line with their desires as being more probable than what is epistemically justified, while assigning lower credence to scenarios they dislike. Striving to avert future dystopias inevitably requires one to contemplate vast amounts of suffering on a regular basis, which is often demotivating and may result in . By contrast, while the prospect of an apocalypse may also be depressing, working towards a utopian future is more inspiring, and could therefore (subconsciously) bias people towards paying less attention to s-risks. Similarly, working towards the reduction of extinction risks or the creation of a posthuman utopia is also favored by many people’s instinctual, self-oriented desires, notably one’s own survival and that of family members or other loved ones. As it is easier to motivate oneself (and others) towards a project that appeals to altruistic as well as more self-oriented desires, efforts to reduce risks of astronomical suffering – risks that lie in the distant future and often involve the suffering of unusual or small minds less likely to evoke empathy – will be comparatively neglected. This does not mean that the above motivations are misguided or unimportant; rather, it means that if one also, upon reflection, cares a great deal about reducing suffering, then it might take deliberate effort to give this concern due justice. Lastly, psychological inhibitions against contemplating s-risks and unawareness of such considerations are interrelated and tend to reinforce each other. II.II Unawareness of possible sources of astronomical suffering In discussions about the risks from smarter-than-human artificial intelligence, it is often assumed that the sole reason to consider AI safety an important focus area is because it decides between utopia or human extinction. The possibility that misaligned or suboptimally aligned AI might instantiate suffering in astronomical quantities is, however, rarely brought up. Misaligned AI as a powerful but morally indifferent optimization process might transform galactic resources into highly optimized structures, some of which might very well include suffering. The structures a superintelligent AI or an AI-based economy decoupled from human interests would build in the pursuit of its goals may for instance include a fleet of “worker bots,” factories, supercomputers to , and space colonization machinery, to name a few. In the absence of explicit concern for suffering reflected in the goals of such an optimization process, AI systems would be willing to instantiate suffering minds (or “”) for even the slightest benefit to their objectives. (Note that, as was the case with natural selection’s use of wild animals, some of these optimization processes might also lead to the instantiation of happy minds.) This is especially worrying because the stakes involved could literally turn out to be astronomical: Space colonization is an attractive subgoal for almost any powerful optimization process, as it leads to control over the largest amount of resources. Even if only a small portion of these resources are used for purposes that involve suffering, the resulting disvalue would tragically be enormous. Finally, next to bad states of affairs being brought about for instrumental reasons, because of indifference to suffering, there is also the risk that bad states of affairs could be brought about for strategic reasons: In competition or conflict between different factions, if one side is compassionate and the other not, threatening to bring about bad states of affairs could be used as an extortion tactic. For an overview on ways the future could contain vast amounts of suffering – including as a result of suboptimally aligned AI or human-controlled futures where dangerous ideologies win – see and . II.III Astronomical suffering as a likely outcome One might argue that the scenarios just mentioned tend to be speculative, maybe extremely speculative, and should thus be discounted or even ignored altogether. However, the claim that creating extremely powerful agents with alien values and no compassion might lead to vast amounts of suffering – through some way or another – is a disjunctive prediction. Only one possible action by which the AI could increase its total utility, yet involving vast quantities of suffering, would be required for the AI to pursue this path without reservation. Worries of this sort are weakly supported by the universe’s historical track record, where the “morally indifferent optimization process” of Darwinian evolution instantiated vast amounts of misery in the form of . Even if the probability of any one specific scenario involving astronomical amounts of suffering (like the ones above, or other scenarios not yet mentioned or thought of) is small, the probability that at least one scenario will occur may be fairly high. In this context, we should beware the disjunction fallacy (Bar-Hillel & Neter, 1993), according to which most people not only underestimate the probability of disjunctions of events, but they actually judge the disjunction as less likely than a single event comprising it.
Rapid advances in AI are coming quickly
Dr. Jolene Creighton 18, Editor-in-Chief at Futurism, Co-Founder of Quarks to Quasars, Ph.D. in Digital Media & Discourse Analysis from University of Southern Mississippi, MA from SUNY Brockport, BA in English Language and Literature/Letters from Keuka College, “The “Father of Artificial Intelligence” Says Singularity Is 30 Years Away”, Futurism, 2-14,
You’ve probably been told that the singularity is coming. It is that long-awaited point in time — likely, a point in our very near future — when advances in artificial intelligence lead to the creation of a machine (a technological form of life?) smarter than humans. If Ray Kurzweil is to be believed, . If we throw our hats in with Louis Rosenberg, then the day will be arriving a little sooner, likely sometime in 2030. MIT’s Patrick Winston would have you believe that it will likely be a little closer to Kurzweil’s prediction, . But what difference does it make? We are talking about a difference of just 15 years. The real question is, is the singularity actually on its way? At the , I spoke with Jürgen Schmidhuber, who is the Co-Founder and Chief Scientist at , Director of the Swiss AI lab IDSIA, and heralded by some as the “father of artificial intelligence” to find out. He is confident that the singularity will happen, and rather soon. Schmidhuber says it “is just 30 years away, if the trend doesn’t break, and there will be rather cheap computational devices that have as many connections as your brain but are much faster,” he said. And that’s just the beginning. Imagine a cheap little device that isn’t just smarter than humans — it can compute as much data as all human brains taken together. Well, this may become a reality just 50 years from now. “And there will be many, many of those. There is no doubt in my mind that AIs are going to become super smart,” Schmidhuber says. Today, the world faces a number of hugely complex challenges, from global warming to the refugee crisis. These are all problems that, over time, will affect everyone on the planet, deeply and irreversibly. But the real seismic change, one that will influence the way we respond to each one of those crises, will happen elsewhere. “It is much more than just another industrial revolution. It is something that transcends humankind and life itself.” “All of this complexity pales against this truly important development of our century, which is much more than just another industrial revolution,” Schmidhuber says. Of course, the development that he is referring to is the development of these artificial superintelligences, a thing that Schmidhuber says “is something that transcends humankind and life itself.” When biological life emerged from chemical evolution, 3.5 billion years ago, a random combination of simple, lifeless elements kickstarted the explosion of species populating the planet today. Something of comparable magnitude may be about to happen. “Now the universe is making a similar step forward from lower complexity to higher complexity,” Schmidhuber beams. “And it’s going to be awesome.” Like with biological life, there will be an element of randomness to that crucial leap between a powerful machine and artificial life. And while we may not be able to predict exactly when, all evidence points to the fact that the singularity will happen.
AI engineers a dystopia worse than extinction.
Toby Ord 20, senior research fellow in philosophy at Oxford University and world-renowned risk-assessment expert who has advised WHO, the World Bank, the WEF, and the US National Intelligence Council, 3/3/2020, The Precipice: Existential Risk and the Future of Humanity, pp. 194-196, cc
Losing our potential means getting locked into a bad set of futures. We can categorize existential catastrophes by looking at which aspects of our future get locked in. This could be a world without humans (extinction) or a world without civilization (unrecoverable collapse). But it could also take the form of an unrecoverable dystopia—a world with civilization intact, but locked into a terrible form, with little or no value.116
[Footnote 116]
In the worst cases, perhaps even negative value—an outcome worse than extinction.
[Footnote 116 ends]
This has not happened yet, but the past provides little comfort. For these kinds of catastrophes only became possible with the advent of civilization, so our track record is much shorter. And there is reason to think that the risks may increase over time as the world becomes more interconnected and experiments with new technologies and ideologies.
I won’t attempt to address these dystopian scenarios with the same level of scientific detail as the risks we’ve explored so far, for the scenarios are diverse and our present understanding of them very limited. Instead, my aim is just to take some early steps toward noticing and understanding these very different kinds of failure.
We can divide the unrecoverable dystopias we might face into three types, on the basis of whether they are desired by the people who live in them. There are possibilities where the people don’t want that world, yet the structure of society makes it almost impossible for them to coordinate to change it. There are possibilities where the people do want that world, yet they are misguided and the world falls far short of what they could have achieved. And in between there are possibilities where only a small group wants that world but enforces it against the wishes of the rest. Each of these types has different hurdles it would need to overcome in order to become truly locked in.
Note that to count as existential catastrophes, these outcomes don’t need to be impossible to break out of, nor to last millions of years. Instead, the defining feature is that entering that regime was a crucial negative turning point in the history of human potential, locking off almost all our potential for a worthy future. One way to look at this is that when they end (as they eventually must), we are much more likely than we were before to fall down to extinction or collapse than to rise up to fulfill our potential. For example, a dystopian society that lasted all the way until humanity was destroyed by external forces would be an existential catastrophe. However, if a dystopian outcome does not have this property, if it leaves open all our chances for success once it ends—it is a dark age in our story, but not a true existential catastrophe.
The most familiar type is the enforced dystopia. The rise of expansionist totalitarianism in the mid-twentieth century caused intellectuals such as George Orwell to raise the possibility of a totalitarian state achieving global dominance and absolute control, locking the world into a miserable condition.117 The regimes of Hitler and Stalin serve as a proof of principle, each scaling up to become imperial superpowers while maintaining extreme control over their citizens.118 However, it is unclear whether Hitler or Stalin had the expansionist aims to control the entire world, or the technical and social means to create truly lasting regimes.119
This may change. Technological progress has offered many new tools that could be used to detect and undermine dissent, and there is every reason to believe that this will continue over the next century. Advances in AI seem especially relevant, allowing automated, detailed monitoring of everything that happens in public places— both physical and online. Such advances may make it possible to have regimes that are far more stable than those of old.
That said, technology is also providing new tools for rebellion against authority, such as the internet and encrypted messages. Perhaps the forces will remain in balance, or shift in favor of freedom, but there is a credible chance that they will shift toward greater control over the populace, making enforced dystopias a realistic possibility.
A second kind of unrecoverable dystopia is a stable civilization that is desired by few (if any) people. It is easy to see how such an outcome could be dystopian, but not immediately obvious how we could arrive at it, or lock it in, if most (or all) people do not want it.120
The answer lies in the various population-level forces that can shape global outcomes. Well-known examples include market forces creating a race to the bottom, Malthusian population dynamics pushing down the average quality of life, or evolution optimizing us toward the spreading of our genes, regardless of the effects on what we value. These are all dynamics that push humanity toward a new equilibrium, where these forces are finally in balance. But there is no guarantee this equilibrium will be good.
For example, consider the tension between what is best for each and what is best for all. This is studied in the field of game theory through “games” like the prisoner’s dilemma and the tragedy of the commons, where each individual’s incentives push them toward producing a collectively terrible outcome. The Nash equilibrium (the outcome we reach if we follow individual incentives) may be much worse for everyone than some other outcome we could have achieved if we had overcome these local incentives.
The most famous example is environmental degradation, such as pollution. Because most of the costs of pollution aren’t borne by the person who causes it, we can end up in a situation where it is in the self-interest of each person to keep engaging in such activities, despite this making us all worse off. It took significant moral progress and significant political action to help us break out of this. We may end up in new traps that are even harder to coordinate our way out of. This could be at the level of individuals, or at the level of groups. We could have nations, ideological blocs, or even planets or descendent species of Homo sapiens locked in harmful competition—doing what is best for their group, but bad for groups on the whole.
I don’t know how likely it is that we suffer a sufficiently bad (and sufficiently intractable) tragedy of the commons like this. Or that we are degraded by evolutionary pressures, or driven to lives of very low quality by Malthusian population dynamics, or any other such situation. I’d like to hope that we could always see such things coming and coordinate to a solution. But it’s hard to be sure that we could.
The third possibility is the “desired dystopia.”121 Here it is easier to see how universal desire for an outcome might cause us to lock it in, though less clear how such an outcome could be dystopian. The problem is that there are many compelling ideas that can radically shape our future —especially ideologies and moral theories, as these make direct normative claims about the world we should strive to create. If combined with the technological or social means for instilling the same views in the next generation (indoctrination, surveillance), this has the potential to be disastrous.
The historical record is rife with examples of seriously defective ideologies and moral views that gripped large parts of the world. Moreover, even reasonable normative views often recommend that they be locked in—for otherwise a tempting rival view may take over, with (allegedly) disastrous results.122 Even though the most plausible moral views have a lot of agreement about which small changes to the world are good and which are bad, they tend to come strongly apart in their recommendations about what an optimal world would look like. This problem thus echoes that of AI alignment, where a strong push toward a mostly correct ideal could instead spell disaster.
Some plausible examples include: worlds that completely renounce further technological progress (which ensures our destruction at the hands of natural risks),123 worlds that forever fail to recognize some key form of harm or injustice (and thus perpetuate it blindly), worlds that lock in a single fundamentalist religion, and worlds where we deliberately replace ourselves with something that we didn’t realize was much less valuable (such as machines incapable of feeling).124
All of these unrecoverable dystopias can be understood in terms of lock-in. Key aspects of the future of the civilization are being locked in such that they are almost impossible to change. If we are locked into a sufficiently bad set of futures, we have an unrecoverable dystopia; an existential catastrophe.
Outweighs everything.
(Max Daniel 17. Executive Director, Foundational Research Institute. 2017. “S-risks: Why they are the worst existential risks, and how to prevent them (EAG Boston 2017).” FRI. )
“S-risk – One where an adverse outcome would bring about severe suffering on a cosmic scale, vastly exceeding all suffering that has existed on Earth so far.” So, s-risks are roughly as severe as factory farming, but with an even larger scope. To better understand this definition, let’s zoom in on the part of the map that shows existential risk. Risks of extinction have received the most attention so far. But, conceptually, x-risks contain another class of risks. These are risks of outcomes even worse than extinction in two respects. First, with respect to their scope, they not only threaten the future generations of humans or our successors, but all sentient life in the whole universe. Second, with respect to their severity, they not only remove everything that would be valuable but also come with a lot of disvalue – that is, features we’d like to avoid no matter what. Recall the story I told in the beginning, but think of Greta’s solitary confinement being multiplied by many orders of magnitude – for instance, because it affects a very large population of sentient uploads.