AI creates weapons that will wipe out humanity

I’ve been researching artificial intelligence and machine learning for more than 30 years. Some people call me one of the “godfathers of AI.” And yet I have grown very concerned that this technology—like the atomic bomb—will grant humanity powers that could destabilize the world. If we do not create policy and guardrails around the use of AI, we could be looking at new and major threats to democracy and humanity.

Right now, AI-powered systems are advanced enough to both pilot drones and use high-powered facial recognition technology to target specific people. There have already been reports of such drone attacks in civilian areas in Syria and Ukraine. This is just the beginning. It is becoming shockingly easy, for example, to use AI-generated code to invent new toxic compounds—potential chemical weapons. Even more concerning to me are biological weapons, which can replicate themselves in the form of viruses or bacteria.

The bad actors here may not necessarily be national armies; they could also be terrorist organizations that have access to open-source large language models similar to ChatGPT, which are trained on a very broad set of documents that includes scientific papers in these areas. Future models could be fine-tuned on chemistry or biological datasets—imagine a publicly available AI system that teaches non-experts how to create chemical or biological weapons.

These military scenarios, though terrifying, assume that the attackers are humans wielding war. My other fear is that, in our race to develop stronger and stronger computer programs, we inadvertently create rogue autonomous AIs with their own self-preservation goals. I fear a scenario where AI develops an objective that supersedes human advancement—like its own proliferation, for example. If that happens, humans will no longer wage war against each other, but against machines. The most dangerous scenario I can imagine is AI that could autonomously fire missiles, destroying property and killing people. This could happen in the next five to 20 years and, at its extreme, it could threaten the survival of humanity.