Rapid improvement in theory of mind tests

(Darren, https://www.popularmechanics.com/technology/robots/a42958546/artificial-intelligence-theory-of-mind-chatgpt/)

The AI revolution is upon us as super-advanced machines continue to master the subtle art of being human at a stunning (concerning?) speed. It’s old news that AI have bested humans at their own games, specifically things like Chess and Go, but there’s more to our brains than checking a king. There are subtler skills like inference and intuition—squishier, almost subconscious concepts that help us understand and predict the actions of others. But with the advent of advanced AI platforms like Open AI’s Generative Pre-training Transformer (GPT), even those boundaries between man and machine are beginning to fade. A new study conducted by Michal Kosinski, a computational psychologist from Stanford University, used several iterations of OpenAI’s GPT neural network—from GPT-1 to the latest GPT-3.5—to perform “Theory of Mind” (ToM) tests, a series of experiments first developed in 1978 to measure the complexity of a chimpanzee’s mind to predict the behavior of others. These tests involve solving normal, mundane scenarios that humans could easily deduce the outcome. For example, one scenario involves mislabeling a bag of popcorn as “chocolate” and then the test asks the AI to infer what the human’s response will be once the bag is opened. Kosinski’s team used “sanity checks” to analyze how well GPT networks understood the scenario and the human’s predicted response. The results were published online on arXiv, the pre-print server. This content is imported from twitter. You may be able to find the same content in another format, or you may be able to find more information, at their web site. While early versions of GPT-1, first released in 2018, scored poorly on the test, the neural network showed stunning improvement stretched across different iterations and spontaneously developed “Theory of Mind” capability of a 9-year-old human by November 2022 (the release of the latest GPT-3.5). Kosinski says this could be a “watershed moment” for AI, as the ability to understand and predict human behavior would make these engines much more useful.