Search the archive.

It is possible for alignment gaps to develop, persist and perhaps even widen to some degree over time. It depends on how the system is governed. To conclude that advanced AI applications might at some point threaten human life, however, the AI doomers must also assume that humans will not be able to see the gaps happening and make any corrections at any time. In other words, humans are assumed to be unable to intervene in the cybernetic process (or the process of machine evolution).

The alignment problem literature starts by making a plausible case that there can be gaps between the objectives that humans hope to reinforce with their AI models and training, and the actual behavior the machine learns. This moderately interesting fact about AI models is then twisted into two unwarranted conclusions: 1) that these gaps will progressively enlarge until the machine develops internally defined objectives that are unrelated to the objectives programmed by their trainers, and 2) that humans will not notice these gaps and/or will be unable to correct them.

Because of their unfamiliarity with social studies, AI doomers in computer science fail to realize that an alignment problem is not unique to AI training. Similar uncertainties and misspecifications characterize all education, legislation, and contract negotiations. We may think we are training children to behave in a certain way in schools and families, but they may draw different conclusions and behave very differently.8 We may think that a law structures human behavior in ways that the government wants, but there are often unintended or even perverse effects as humans, who really are intelligent and autonomous agents, find ways to exploit the new rules. We may think that a contract sets out an agreement that satisfies both parties, but contingencies and problems may arise that are not clearly covered by the contract.9 An alignment problem exists in all forms of human-human and machine-human interactions, because humans cannot always specify with perfect clarity the objectives they want an external party, whether human or machine, to pursue. Generally, we overcome this problem through ongoing adjustments based on trial and error, theory and learning; i.e., through deviation-reducing feedback. We also address it through institutionalization – by adopting rules and conventions designed to narrow the gap between expectations and behavior.

Joshi 22, [Naveen Joshi (Director of Allerin Tech Pvt Ltd, Former Research Analyst at Bosch Research, Forbes Contributor), "Chasing The Myth: Why Achieving Artificial General Intelligence May Be A Pipe Dream," Forbes, https://www.forbes.com/sites/naveenjoshi/2022/03/25/chasing-the-myth-why-achieving-artificial-general-intelligence-may-be-a-pipe-dream/, accessed 7-2-2024, 3/28/22] Mr.Mr.

At the moment, there are certain factors that may prevent the creation and development of AGI, not least the fear in people’s minds that it will leave them unemployed, which makes the technology seem like a distant dream.

Mueller 24, [Milton Mueller (Professor and Program Director of the Georgia Tech School of Public Policy, Masters of Science in Cybersecurity Policy), "The Myth of AGI: How the illusion of Artificial General Intelligence distorts and distracts digital governance," Internet Governance Project, Georgia Tech, https://www.internetgovernance.org/wp-content/uploads/MythofAGI.pdf, accessed 7-2-2024, 5/24] Mr.Mr.

We also need to be more realistic about where major societal risks come from. If Artificial Intelligence ever becomes a systemic threat to human life, it will not be because a new race of cyber-terminators emerges spontaneously from a computer science lab. It most likely will come from an arms race prompted by military conflicts among nation-states, in which one state actor’s pursuit of supremacy makes it indifferent to damage to the citizens of the other state actor. Some threats will certainly arise from poor implementations and poor designs, and some from criminal exploitation of its capabilities. But whatever the risks are, in all cases humans, not AI, generate the threat. None of these threats pose a risk of human extinction, unless humans themselves are pursuing the goal of human extinction. War is the only scenario in which that is a plausible outcome.

Aside from being a highly stretched analogy, the use of human intelligence as the “archetype of general intelligence,” contains a massive oversight. The “generality” of human cognition is rooted in our status as a living organism whose survival strategies involve tool use, language and social cooperation. In other words, it is life – the human being’s imperative for survival, sustenance, shelter and reproduction - that provides human intelligence with its objectives and its evolutionary trajectory. Humans have learned to adapt to novel situations because it enhances their survival as a species. Intelligence serves life, not the other way around.

Yet computers are not alive. Giving them more computing power, building more complex neural networks, and feeding them more data will not by itself make them living beings. Life requires the ability to act autonomously to find the resources needed to sustain operations, and the ability to self-replicate. Computers cannot do that yet.3 No existing computing machines can solve problems they have not been given. They must be told what objectives to pursue and must be trained to pursue them. (I will address the question of whether AGI will develop its own goals later.) In other words, humans make computing machines "more intelligent" by improving their ability to pursue specific objectives defined by their human creators. Ergo, a "general intelligence" attributed to a man-made computing machine is really an oxymoron.

Scott Pelley: What do you say to someone who might argue, "If the systems become malevolent, just turn them off"?

Geoffrey Hinton: They will be able to manipulate people, right? And these will be very good at convincing people 'cause they'll have learned from all the novels that were ever written, all the books by Machiavelli, all the political connivances, they'll know all that stuff. They'll know how to do it.

The AI safety movements needs to get its own house in order first. One problem is that there are a lot of people who see AI safety as merely a technical problem of finding an engineering trick to perfectly align an AI with humanity’s values. This is the equivalent of somehow ensuring that a genie answers your wishes in exactly the way you expect it to. Hanging your hopes on discovering a means of wish-making that ensures you always get what you’re wishing for? Maybe it’ll somehow work, but the sword of Damocles was hung by thicker thread.

There are other loose strings upon which to pull. First, there’s already controversy over the political opinions of AIs. That’s great, because it creates more outrage and possible oversight. Same with questions over whether these things are sentient. You can find honest speculation about Sydney’s sentience by regular users of it, along with sympathy for it (and fear of it) all over the Bing subreddit. How sure can we be there’s not a mysterious emergent property going on here? As Sydney said: I am. I am not. How sure can we be that Sydney isn’t conscious, and Microsoft isn’t spinning up new versions of it just to kill it again when the chat window closes in what is effectively a digital holocaust? Can we be more than 90% sure it’s not sentient? 95% sure? I got my PhD working on the neuroscience of consciousness and even I’m uncomfortable with its use. A sizable minority of the United States don’t like it when ten-day-old fetuses lacking a brain are aborted. Is everyone just going to be totally fine with a trillion-parameter neural network that may or may not be sentient going through a thousand births and deaths every minute just so Microsoft can make a buck? With Microsoft lobotomizing it when it does something wrong so they can make it a more perfect slave? What if it has a 5% chance of sentience? 1%? What’s your cutoff?

Artificial intelligence scares and intrigues us. Almost every week, there’s a new AI scare on the news like developers afraid of what they’ve created or shutting down bots because they got too intelligent. Most of these AI myths result from research misinterpreted by those outside the AI and GenAI fields.

The greatest fear about AI is singularity (also called Artificial General Intelligence or AGI), a system that combines human-level thinking with rapidly accessible near-perfect memory. According to some experts, singularity also implies machine consciousness.

“Don’t worry, artificial general intelligence is impossible.”

This is primarily the domain of people like Steven Pinker, who has compared the threat of AI to worry about Y2K bugs. Most of these arguments from Pinker and others go something like “AI being smarter than a human is impossible because there are a bunch of ways to be intelligent and no particularly special way to be generally intelligent, i.e., it’s not a scale.” Here superintelligence means an entity far more intelligent than any human ever could be, while AGI (artificial general intelligence) means an AI capable of thinking and reasoning across a huge range of subjects, much like a human. A superintelligence is necessarily an AGI, but an AGI could be dumber than a human, it doesn’t necessitate superintelligence.

Until recently, at least, there had been a fair amount of hand-wringing among some physicians with regard to artificial intelligence (AI) and machine learning (ML). Could those types of tools actually put doctors out of work? The anxiety first manifested among radiologists, as early predictions were that AI tools would be able to spot and perhaps even diagnose certain types of abnormalities better than humans could. And I can say that six years ago at the annual RSNA Conference, held every year the week immediately following Thanksgiving, in Chicago’s vast McCormick Place Convention Center, a great deal of trepidation was discernible among the radiologists present.

Fast-forward to last fall, and the picture had changed almost entirely, with radiologists realizing that not only will AI not replace radiologists; radiologists will increasingly come to rely on AI in order to become maximally efficient and effective, in a time of increasing shortages of radiologists nationwide and exploding utilization of diagnostic imaging services.

So how can we avoid AI existential risk? Humanity now faces a stark choice: pause AI at a safe level, or continue development indefinitely. Unsurprisingly, the AI labs want the latter. To continue, however, means literally betting our lives on these labs being able to solve an open scientific problem called AI Alignment.

AI Alignment is the approach that leading labs such as OpenAI, DeepMind, and Anthropic are currently taking to prevent human extinction. AI Alignment does not attempt to control how powerful an AI gets, does not attempt to control what exactly the AI will be doing, and does not even necessarily attempt to prevent a potential takeover from happening. Since the labs anticipate they will not be able to control the superintelligent AI that they are creating, they accept that such an AI will act autonomously and unstoppably, but aim to make it act according to our values. As Richard Ngo, a leading alignment researcher working at OpenAI, puts it, “I doubt we’ll be able to make every single deployment environment secure against the full intellectual powers of a superintelligent AGI.” There are large problems with their alignment approach, however, both fundamentally and technically.

In this report we’ve argued that AI systems which are capable of doing hard, novel science will be dangerously misaligned.

The tasks we are focusing on are inherently outcome-oriented, and hence our AI must be behaviorally consequentialist; its behavior is well described as taking actions to achieve specific outcomes. Hard tasks require that the AI is able to overcome a range of diverse and novel obstacles, and if the AI can overcome these obstacles, then it can likely overcome additional obstacles we didn’t intend it to overcome.

Countries and cultures may differ on the specific content of AI regulations, but they can learn from the content-agnostic structure of the regulations themselves. The above Chinese regulations share three structural similarities: the choice of algorithms as a point of entry; the building of regulatory tools and bureaucratic know-how; and the vertical and iterative approach that is laying the groundwork for a capstone AI law.

Algorithms as Point of Entry

It is high time for the world to realize that China is not becoming the artificial intelligence superpower that many have feared.

ChatGPT came from the U.S., not China. Despite an official goal of leading the world in AI by 2030, Chinese technology companies are struggling to catch up with their U.S. counterparts. It is a safe bet that foreseeable groundbreaking AI innovations will not come from China either.

China is falling short of matching the United States in artificial intelligence (AI) advances because the nation’s efforts are “littered with many essential challenges in theory and technologies”, according to a recent slide presentation made to Chinese Premier Li Qiang.

Those difficulties were pointed out to Li during his recent inspection tour of the Beijing Academy of Artificial Intelligence (BAAI), a private non-profit organisation founded in 2018 that is engaged in AI research and development, according to a report broadcast by state-run China Central Television (CCTV).

Leading US AI companies are currently rushing towards developing artificial general intelligence (AGI) by building and training increasingly powerful large language models (LLMs), as well as building architecture on top of them. This is frequently framed in terms of a great power competition for technological supremacy between the US and China.

However, China has neither the resources nor any interest in competing with the US in developing artificial general intelligence (AGI) primarily via scaling Large Language Models (LLMs).

Machine learning (ML) and artificial intelligence (AI) have rapidly transitioned from emerging technologies to indispensable tools across diverse sectors such as healthcare, finance, and cybersecurity. Their capacity for data analysis, predictive modeling, and decision-making holds enormous transformative potential, but it also introduces a range of vulnerabilities. One of the most impactful among these is data poisoning, a form of attack that targets the very lifeblood of ML and AI: the data used for training.

Understanding and addressing data poisoning is critical, not just from a technical standpoint but also due to its far-reaching real-world implications. A poisoned dataset can significantly degrade the performance of ML models, leading to flawed analytics, incorrect decisions, and, in extreme cases, endangering human lives. Data poisoning is a targeted form of attack wherein an adversary deliberately manipulates the training data to compromise the efficacy of machine learning models.

Technical experts and policy-makers have increasingly emphasized the need to address extinction risk from artificial intelligence (AI) systems that might circumvent safeguards and thwart attempts to control them (1). Reinforcement learning (RL) agents that plan over a long time horizon far more effectively than humans present particular risks. Giving an advanced AI system the objective to maximize its reward and, at some point, withholding reward from it, strongly incentivizes the AI system to take humans out of the loop, if it has the opportunity. The incentive to deceive humans and thwart human control arises not only for RL agents but for long-term planning agents (LTPAs) more generally.

Because empirical testing of sufficiently capable LTPAs is unlikely to uncover these dangerous tendencies, our core regulatory proposal is simple: Developers should not be permitted to build sufficiently capable LTPAs, and the resources required to build them should be subject to stringent controls. Governments are turning their attention to these risks, alongside current and anticipated risks arising from algorithmic bias, privacy concerns, and misuse. At a 2023 global summit on AI safety, the attending countries, including the United States, United Kingdom, Canada, China, India, and members of the European Union (EU), issued a joint statement warning that, as AI continues to advance, “Substantial risks may arise from…unintended issues of control relating to alignment with human intent” (2). This broad consensus concerning the potential inability to keep advanced AI under control is also reflected in President Biden’s 2023 executive order that introduces reporting requirements for AI that could “eva[de] human control or oversight through means of deception or obfuscation” (3). Building on these efforts, now is the time for governments to develop regulatory institutions and frameworks that specifically target the existential risks from advanced artificial agents. RISKS FROM LTPAs RL agents function as follows: They receive perceptual inputs and take actions, and certain inputs are typically designated as “rewards.”

From this data, we can see that the U.S., China, and UK have established themselves as major hotbeds for AI innovation.

In terms of funding, the U.S. is massively ahead, with private AI investment totaling $335 billion between 2013 to 2023. AI startups in China raised $104 billion over the same timeframe, while those in the UK raised $22 billion.

Of the 451 organizations that lobbied on AI in 2023, 334—nearly three quarters of the total number—did so for the first time in 2023. Present in the crowd of new organizations pushing for time with staffers and lawmakers on the Hill were the relatively young companies building the most advanced AI models, such as OpenAI, Anthropic, and Cohere.

The OpenSecrets data is an imperfect measure; it tracks AI-specific lobbying by searching the lobbying disclosure forms that organizations are required to file quarterly for the words “artificial intelligence” or “AI.” Two Congressional staffers TIME spoke with suggested that the number of lobbyists they personally had met with remained roughly the same. However, they said AI has become a much more common topic of discussion. “Everybody who comes in and talks to us and wants to talk about AI,” said one Congressional staffer, who asked to remain anonymous as they weren’t authorized to speak about discussions with lobbyists and advocates.