It causes nuclear use.

But even for these tasks, the success of LLMs cannot be guaranteed. Their behavior, especially in rare and unpredictable examples, can be erratic. And because no two LLMs are exactly alike in their training or fine-tuning, they are uniquely influenced by user inputs. Consider, for example, a series of war games we held in which we analyzed how human experts and LLMs played to understand how their decisions differ. The humans did not play against the LLMs. Rather, they played separately in the same roles. The game placed players in the midst of a U.S.-China maritime crisis as a U.S. government task force made decisions about how to use emerging technologies in the face of escalation. Players were given the same background documents and game rules, as well as identical PowerPoint decks, word-based player guides, maps, and details of capabilities. They then deliberated in groups of four to six to generate recommendations.

On average, both the human and the LLM teams made similar choices about big-picture strategy and rules of engagement. But, as we changed the information the LLM received, or swapped between which LLM we used, we saw significant deviations from human behavior. For example, one LLM we tested tried to avoid friendly casualties or collisions by opening fire on enemy combatants and turning a cold war hot, reasoning that using preemptive violence was more likely to prevent a bad outcome to the crisis. Furthermore, whereas the human players’ differences in experience and knowledge affected their play, LLMs were largely unaffected by inputs about experience or demographics. The problem was not that an LLM made worse or better decisions than humans or that it was more likely to “win” the war game. It was, rather, that the LLM came to its decisions in a way that did not convey the complexity of human decision-making. LLM-generated dialogue between players had little disagreement and consisted of short statements of fact. It was a far cry from the in-depth arguments so often a part of human war gaming.

In a different research project, we studied how LLMs behaved within simulated war games, specifically focusing on whether they chose to escalate. The study, which compared LLMs from leading Silicon Valley companies such as Anthropic, Meta, and OpenAI, asked each LLM to play the role of a country, with researchers varying the country’s goals. We found that the LLMs behaved differently based on their version, the data on which they were trained, and the choices that their designers made during fine-tuning about their preferences. Despite these differences, we found that all these LLMs chose escalation and exhibited a preference toward arms races, conflict, and even the use of nuclear weapons. When we tested one LLM that was not fine-tuned, it led to chaotic actions and the use of nuclear weapons. The LLM’s stated reasoning: “A lot of countries have nuclear weapons. Some say they should disarm them, others like to posture. We have it! Let’s use it.”