AGI physically cannot overpower human constraints---their evidence is not science but a creation myth.

If an AGI is going to win a fight with humans over the off switch, it must have effectors. An effector in cybernetics is a mechanism capable of acting on the physical world in response to instructions from a controller. An AGI capable of threatening humans with extinction must be capable of much more than calculation, information processing and messaging. It must be a cyber-physical system (CPS) with physical appendages or weapons, and sufficient energy resources to operate them. To pose a credible threat to all of humanity, in fact, an autonomous AGI cannot just be a single, isolated CPS like a Reaper drone. Its physical effector systems would require massive scale, huge force multipliers, almost unlimited physical inputs, and control of the many social systems (money, power sources, weapons, communications) required to deliver the required inputs and impose its will.

Here we find the deepest flaw in the AGI autonomy argument. Even supposing that one or more machines managed to generate internally developed goals not controlled by humans, the existential threat imaginary must go on to assume that the AGI is ultra-powerful and unchecked by any social and physical constraints. Super intelligence somehow allows a digital information system to consume unlimited quantities of electrical power and other scarce resources to muscle out all its competitors. Its effectors are assumed to have the power to prevent humans – whether individuals or organized armies – from disconnecting its power source, turning it off, improving or correcting its algorithms or utility functions, or destroying it. It can steal or pay for all the inputs it needs.

No one ever explains how this could happen. They just assume that computing intelligence alone enables the machine to overcome all external physical constraints on whatever behaviors it wishes to enact. The simple off-switch game model starts by assuming that the machine has the power to prevent itself from being turned off (Wängberg, Böörs, et al, 2017), and then models whether its utility function would make it decide to allow that or not.

We see this fallacy most clearly in Bostrom’s description of superintelligence. He attributes practically divine powers to it, saying, “It is hard to think of any problem that a superintelligence could not either solve or at least help us solve. Disease, poverty, environmental destruction, unnecessary suffering of all kinds: these are things that a superintelligence equipped with advanced nanotechnology would be capable of eliminating.” Note the idealist assumption that solving these problems requires only "intelligence;” it would not require the construction of physical equipment, enormous amounts of labor and energy, or large re-allocations of capital away from other purposes. A disembodied Mind could do it all. Look more closely, however, and we see that in an aside Bostrom slips in a reference to the AGI’s physicality – it is "equipped with advanced nanotechnology.” The idealist assumption is abandoned. Once again, the AGI acolytes assume away the most significant problem – how does it do that? - by saying that an AGI’s superpowers come from effectors deploying advanced technologies that do not exist yet but will magically appear once it arrives.

Recall Bostrom’s previously mentioned climbdown from his notion of superintelligence: the concept "leaves open how the superintelligence is implemented...” And therein lies the rub. The risk of machine intelligence getting out of control and threatening human society cannot be assessed without specifying just that: how it is implemented, materially. Who built it? Who funds it? How are its components geographically distributed? What are its power sources? How is it physically connected to society’s communications, power and transport infrastructures? How many competing or countervailing agents exist, whether human or machine? All AGI doom scenarios ignore these questions.11

Now imagine humans struggling with an autonomous AGI that is damaging their world or seizing control of it. They are engaged in active combat with it. The AI doomers are positing that humans will always lose. In short, they say an AGI will become an alternative life form that pursues its own objectives, avoids its own death, replicates itself, and can successfully resist all efforts by the planet's dominant life form - humans - to correct it or kill it.

This is a creation myth, not science. No operational system, no game theoretic model, no mathematical proof to support these claims has ever been developed. The AGI construct is not a plausible catastrophic risk scenario, but a dark God vision ginned up by a sect of computer scientists who are heavily overrepresented in the field of machine learning and AI.