Rhetoric of an AI arms race with China accelerates a dangerous cycle that prevents effective regulation and results nuclear catastrophe.

How does international competition come into play when discussing the technical issue of alignment? Put simply, the faster AI advances, the less time we will have to learn how to align it. The alignment problem is not yet solved, nor is it likely to be solved in time without slower and more safety-conscious development.

The fear of losing a technological arms race may encourage corporations and governments to accelerate development and cut corners, deploying advanced systems before they are safe. Many top AI scientists and organizations — among them the team at safety lab Anthropic, Open Philanthropy’s Ajeya Cotra, DeepMind founder Demis Hassabis, and OpenAI CEO Sam Altman — believe that gradual development is preferable to rapid development because it offers researchers more time to build safety features into new models; it is easier to align a less powerful model than a more powerful one.

Furthermore, fears of China’s “catching up” may imperil efforts to enact AI governance and regulatory measures that could slow down dangerous development and speed up alignment. Altman and former Google CEO Eric Schmidt are on record warning Congress that regulation will slow down American companies to China’s benefit. A top Microsoft executive has used the language of the Soviet missile gap. The logic goes: “AGI is inevitable, so the United States should be first.” The problem is that, in the words of Paul Scharre, “AI technology poses risks not just to those who lose the race but also to those who win it.”

Likewise, the perception of an arms race may preclude the development of a global governance framework on AI. A vicious cycle may emerge where an arms race prevents international agreements, which increases paranoia and accelerates that same arms race.

International conventions on the nonproliferation of nuclear bombs and missiles and the multilateral ban on biological weapons were great Cold War successes that defused arms races. Similar conventions over AI could dissuade countries from rapidly deploying AI into more risky domains in an effort to increase national power. More global cooperation over AI’s deployment will reduce the risk that a misaligned AI is integrated into military — and even nuclear — applications that would give it a greater capacity to create a catastrophe for humanity.