US industry capture results in misalignment and paper regulation.

The most unusual thing about this week’s Senate hearing on AI was how affable it was. Industry reps — primarily OpenAI CEO Sam Altman — merrily agreed on the need to regulate new AI technologies, while politicians seemed happy to hand over responsibility for drafting rules to the companies themselves. As Senator Dick Durbin (D-IL) put it in his opening remarks: “I can’t recall when we’ve had people representing large corporations or private sector entities come before us and plead with us to regulate them.”

This sort of chumminess makes people nervous. A number of experts and industry figures say the hearing suggests we may be headed into an era of industry capture in AI. If tech giants are allowed to write the rules governing this technology, they say, it could have a number of harms, from stifling smaller firms to introducing weak regulations.

Experts at the hearing included IBM’s Christina Montgomery and noted AI critic Gary Marcus, who also raised the specter of regulatory capture. (The peril, said Marcus, is that “we make it appear as if we are doing something, but it’s more like greenwashing and nothing really happens, we just keep out the little players.”) And although no one from Microsoft or Google was present, the unofficial spokesperson for the tech industry was Altman.

Although Altman’s OpenAI is still called a “startup” by some, it’s arguably the most influential AI company in the world. Its launch of image and text generation tools like ChatGPT and deals with Microsoft to remake Bing have sent shockwaves through the entire tech industry. Altman himself is well positioned: able to appeal to both the imaginations of the VC class and hardcore AI boosters with grand promises to build superintelligent AI and, maybe one day, in his own words, “capture the light cone of all future value in the universe.”

At the hearing this week, he was not so grandiose. Altman, too, mentioned the problem of regulatory capture but was less clear about his thoughts on licensing smaller entities. “We don’t wanna slow down smaller startups. We don’t wanna slow down open source efforts,” he said, adding, “We still need them to comply with things.”

Sarah Myers West, managing director of the AI Now institute, tells The Verge she was suspicious of the licensing system proposed by many speakers. “I think the harm will be that we end up with some sort of superficial checkbox exercise, where companies say ‘yep, we’re licensed, we know what the harms are and can proceed with business as usual,’ but don’t face any real liability when these systems go wrong,” she said. 

Other critics — particularly those running their own AI companies — stressed the potential threat to competition. “Regulation invariably favours incumbents and can stifle innovation,” Emad Mostaque, founder and CEO of Stability AI, told The Verge. Clem Delangue, CEO of AI startup Hugging Face, tweeted a similar reaction: “Requiring a license to train models would be like requiring a license to write code. IMO, it would further concentrate power in the hands of a few & drastically slow down progress, fairness & transparency.”