The regulatory question The increased capability and competitiveness of open-source models are running headlong into the unresolved question of how the U.S. government should regulate these AI models. The ad hoc directive, issued by the Commerce Department in June, which forced Anthropic to retract its most advanced model showed that the Trump administration is worried about cyber risks posed by these models —but has not yet developed a clear procedure or benchmark that would give the administration confidence that a model is safe to release to the public. It is, of course, much more difficult for the U.S. government to regulate open-source models. If they come from China, the U.S. can’t dictate the terms of their release. If they’re developed in the U.S., the open-source models are still diffused much more rapidly and can be “fine-tuned” by bad actors. There’s also another distinct threat posed by Kimi and the other leading Chinese open models, such as DeepSeek and Alibaba: that Chinese AI developers have embedded security vulnerabilities into the models’ architecture, so that they can be exploited once they become embedded in American companies and critical infrastructure. “The Commerce Department’s [Center for AI Standards and Innovation] has put out four reports on Chinese AI models, where they include assessments of their capabilities and some of their risks, particularly prompt injection, agent hijacking, [and] security vulnerabilities writ large,” said Daniel Remler, an analyst at the Center for a New American Security. “They make them paint a pretty compelling case that these models are significantly more insecure than the U.S. models,” said Remler, who also argued that the government should develop minimum testing standards that the models should pass before they’re integrated into U.S. infrastructure. The national security question Open-source is a competitive threat to the U.S. frontier AI models. There are catastrophic biological and cyber risks posed by frontier models, both open and closed. And there are unique national security risks posed by open Chinese models. The administration has not developed a substantive, transparent procedure for dealing with these risks. But there is significant disagreement as to whether the U.S. government and the ecosystem at large are capable of making open-source models safe enough, whether through regulatory action or technical breakthroughs. “I think it’s pretty clear that we are approaching the point I describe — the point where, absent a major technical safety breakthrough, the national security implications of frontier open-weight model distribution are simply too severe,” said Ball in a follow-up post on X. “Governments will realize these risks eventually, and when they do, they will have much lower risk tolerance.” Not everyone disagrees with Ball’s assessment. Kristian Stout, director of innovation policy at the International Center for Law & Economics, told me in an interview Monday that he believes, from a catastrophic risk perspective, banning open models hurts more than it helps. Stout argues specifically that you need frontier AI defense capabilities to defend against AI-powered cyberattacks. “I need an advanced AI agent that’s sitting on my computer that goes out and scans the dark web all day long looking for people who are trying to use my identity or compromise me,” argues Stout. “That only happens when you have more intelligence diffusing.” The outlook: From an economic perspective, though, most AI industry analysts agree that the rise of open-source poses a very real threat to the leading U.S. labs. If open-source survives, whether in the form of U.S. or Chinese models, the enormous valuations of OpenAI and Anthropic become more difficult to justify. Moonshot AI’s Kimi was ranked as better at coding than those other companies’ models, and most companies don’t need the very best model for everyday tasks anyway. Ball and Samuel Hammond, the director of artificial intelligence policy at the Foundation for American Innovation, also make the point that open-source models are inherently “decelerationist”: They cut into the gross margin that frontier labs would have used to reinvest back into AI model training. That could slow down under open-source competition. And, in the latest warning sign for the U.S. frontier labs, Chinese multinational Alibaba announced on X Sunday that it would be launching its newest model, Qwen 3.8, soon. The model has 2.4 trillion parameters and would be second only in capability to Anthropic’s Fable 5 model, the post said. “Value shifts in the stack. It’ll shift away from OpenAI and Anthropic. It’ll shift toward value-added companies, which is what you have seen in every technical revolution where there’s been a general-purpose technology like electricity,” argued Stout. “We live in a world now where electricity is just distributed, and the real value of electricity is what all the complement producers do with electricity,” Stout said. “I believe that is what the intelligence revolution is going to be like.” The most prominent obstacle getting in the way of that economic future is regulation. And the size and scope of the regulation will be determined by which arguments about catastrophic and national security risk win out. This newsletter is published by WP Intelligence, The Washington Post’s subscription service for professionals that provides business, policy and thought leaders with actionable insights. WP Intelligence operates independently from The Washington Post newsroom. Learn more about WP Intelligence. |