A super-intelligent AI is uncontrollable.
Robby Berman, 2021 (political analyst), THE FUTURE. Jan. 20, 2021. Retrieved May 18, 2022 from https://bigthink.com/the-future/superintelligent-ai-planck/
There have been a fair number of voices—Stephen Hawking among them—raised in warning that a super-intelligent artificial intelligence could one day turn on us and that we shouldn’t be in such a hot, unquestioning hurry to develop true AI. Others say, naw, don’t worry. Now a new white paper from scientists at the Center for Humans and Machines at the Max Planck Institute for Human Development presents a series of theoretical tests that confirm the threat: Due to the basic concepts underlying computing, we would be utterly unable to control a super-intelligent AI. “We argue that total containment is, in principle, impossible, due to fundamental limits inherent to computing itself,” write the paper’s authors.