Free beta access. 50 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Metrics on Chinese AI prowess misleading

Toner, Georgetown University Center for Security and Emerging Technology Director of Strategy and Foundational Research Grants, et al., 23

[Helen Toner, Jenny Xiao, AI venture capitalist, Jeffrey Ding, George Washington University Political Science Assistant Professor, 2 June 2023, Foreign Affairs, “The Illusion of China’s AI Prowess,” https://www.foreignaffairs.com/china/illusion-chinas-ai-prowess-regulation?check_logged_in=1&utm_medium=promo_email&utm_source=lo_flows&utm_campaign=registered_user_welcome&utm_term=email_1&utm_content=20230629 , 6/2923, EJ]

Take, for instance, the Beijing Academy of Artificial Intelligence’s WuDao 2.0 model. After its release in the summer of 2021, Forbes thrilled at the model as an example of “bigger, stronger, faster AI,” largely because WuDao 2.0 boasted ten times more parameters—the numbers inside an AI model that determine how it processes information—than GPT-3. But this assessment was misleading in several ways. Merely having more parameters does not make one AI system better than another, especially if not matched by corresponding increases in data and computing power. In this case, comparing parameter counts was especially unwarranted given that WuDao 2.0 worked by combining predictions from a series of models rather than as a single language model, a design that artificially inflated the parameter count. What’s more, the way researchers posed questions to the model helped its performance in certain trials appear stronger than it actually was.

A fear of Chinese AI prowess should not stop the United States from regulating the technology.

Baidu’s “Ernie Bot” also disappointed. Touted as China’s answer to ChatGPT, the development of Ernie Bot was clearly—like that of WuDao 2.0—spurred by pressure to keep up with a high-profile breakthrough in the United States. The Chinese bot failed to live up to aspirations. Baidu’s launch event included only prerecorded examples of its operation, a telltale sign that the chatbot was unlikely to perform well in live interactions. Reviews from users who have since gained access to Ernie Bot have been mediocre at best, with the chatbot stumbling on simple tasks such as basic math or translation questions.

Sign in to recut, update, or save this card.