Free beta access. 50 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Safeguards will be circumvented by the AI itself.

Shaun Tan, 9/26/2017 (staff writer, “We Need an AI Limitation Treaty. Now.” https://www.chinausfocus.com/society-culture/we-need-an-ai-limitation-treaty-now, Retrieved 6/15/2022)

Then, there’s the likelihood that any safeguards would be circumvented by the AI itself. Indeed, any security measures our best computer scientists could devise would be laughable to a superintelligence, which by definition would be many times smarter than any human. Imagine a human being held captive by chimpanzees. Suppose that these are unusually intelligent chimpanzees that use state-of-the-art monkey technology to keep the human prisoner – perhaps they manage to construct a rudimentary cage out of sticks. Is there any doubt that the human wouldn’t eventually escape in ways the chimpanzees couldn’t possibly think of? Perhaps he’d dig a hole under the cage, or fashion tools out of nearby objects to help him, or remove the bars of the cage and use them as weapons, or make a fire that burns down a portion of the cage. One way or another, it would only be a matter of time before he found a way free. A superintelligence would be smarter than humans in a similar fashion. In his article “Leakproofing the Singularity: Artificial Intelligence Confinement Problem,” Yampolskiy suggested that a superintelligence could easily manipulate a human guard into letting it escape. It could target a guard’s weaknesses, offering him power or immortality, or promising a cure for a loved-one with a terminal disease. It could also find a bug in the system and exploit it (something even human hackers do all the time). Or pretend to malfunction, and then escape when its jailors lower safeguards to investigate. Or it could escape in ways humans aren’t even aware are possible. Insulated from the outside world, Bostrom suggested, it might find a way to generate radio waves by shuffling the electrons in its circuitry in particular patterns. Of course, these are just the methods our puny human brains can imagine – an entity thousands of times smarter would be able to come up with a lot more. Effective safeguards are built around power – they’re not possible against a being that’s smarter, and therefore more powerful, than us. Thinking we could contain something like that would be hubris.

Sign in to recut, update, or save this card.