Attempts to control AI will backfire and risk extinction of the human race.
Manuel Alfonseca, et al, 2021 (Escuela Politécnica Superior, Universidad Autónoma de Madrid, Madrid, Spain), JOURNAL OF ARTIFICIAL INTELLIGENCE RESEARCH. 2021. Retrieved May 18, 2022 from https://www.jair.org/index.php/jair/article/view/12202/26642
Bostrom extensively discusses the weaknesses of the various mechanisms. He relies on sce-narios in which, short of rendering the AI useless, well-intentioned control mechanisms can easily backfire. As an illustrative example, a superintelligence given the task of “maximizing happiness in the world,” without deviating from its goal, might find it more efficient to destroy all life on earth and create faster-computerized simulations of happy thoughts. Likewise, a superintelligence con-trolled via an incentive method may not trust humans to deliver the promised reward, or may worry that the human operator could fail to recognize the achievement of the set goals.