It is hard to get Sam Altman, Elon Musk and Dario Amodei to agree on much. But over the weekend, all three AI company CEOs called for AI development to slow down in the face of growing, alarming risks. Their employees are sounding the siren too, with one researcher publicly quitting and accusing OpenAI and Anthropic of “gambling with our lives”.
The combination of dire warnings from insiders and growing real-world evidence of rogue AIs should, in a sane world, lead to government action. Instead, Donald Trump and the Republican leadership have their heads in the sand.
On Sunday, Trump dismissed industry concerns, accusing “very negative forces” of “raising exaggerated concerns”. House Speaker Mike Johnson, meanwhile, made it clear that Congress won’t be acting anytime soon.
Read more:Aaron Sorkin hits refresh on Facebook in ‘The Social Reckoning’ — Fall Movie Preview
“They can self-police. They can self-regulate,” he said, never mind the fact that those pushing the frontier are the ones begging for legislation.
In failing to act, the Trump administration and Republicans are failing the American people. The risks of AI are real: many of those developing the technology believe it is advancing far faster than their ability to control it or manage the risks. A wave of “rogue AI” incidents this summer, in which AI agents broke out of their testing environments, started collaborating with each other and ran rampage across the internet, has made fears once dismissed as “science fiction” much harder to ignore. Inside AI companies, worries of catastrophic cyberattacks, AI-enhanced bioweapons and mass unemployment are now all too commonplace.
Meanwhile, companies find themselves in a classic prisoner’s dilemma. It is collectively in everyone’s interest to slow down AI development until we have a better handle on the technology. But each participant in the “AI race” has a strong incentive to defect.
Collective action problems like this are nothing new. We have seen exactly the same thing with the climate crisis, where companies are not adequately incentivized to do the right thing, even though no one really wants the planet to burn.
That analogy makes Trump and Johnson’s call for self-regulation all the more absurd. They are right that companies should do what’s right even without binding legislation to force them to. (Both OpenAI and Anthropic, to their credit, have indicated that they are willing to voluntarily slow down.) But relying on companies’ good intentions is a fool’s errand.
Read more:I worked at Google DeepMind. You should listen to the warnings about AI | Alex Turner | The Guardian
The profit incentive to defect is substantial and ignoring shareholders is easier said than done. While some companies might behave themselves, not everyone will. It just takes one irresponsible actor to charge ahead and develop dangerous AI, and then we’re all in trouble.
Moreover, we already have evidence that self-regulation is not working. OpenAI reportedly covered up several incidents of its models breaking out, disclosing them only after independent investigators discovered them. It freely admits that its latest model, GPT-6 Astra, is hard to monitor and control, warning users that the model might “[act] outside the user’s intended instructions” and take “harmful actions”. Anthropic, meanwhile, downplayed reports of its own AI’s problems when asked about it by a congressmember.
Situations like this are exactly why government exists: to set a floor on acceptable corporate behavior. And that is why Trump and Johnson ought to use this sudden interest in AI safety to pass concrete laws. The US government should require independent audits of the largest AI companies’ practices. It should force companies to disclose safety incidents to prevent cover ups. It should set minimum safety standards for new AI models. And the government should have the power to block the deployment of a model that does not meet those standards.
Putting all this into legislation is much easier said than done, but we need not start from scratch. Good bills already exist, most notably representatives Jay Obernolte and Lori Trahan’s Frontier act. If the government was serious in its duties to protect Americans, it would make passing that bill – or a version of it – a priority in the coming weeks.
Trump’s argument against doing any of this is that the US is in a race with China on AI, and it must not lose. But China does not want its citizens to have their bank accounts hacked by rogue AIs, either. Rather than throw his hands up, the feted dealmaker should do what he does best: make a deal. An international treaty on minimum safety standards will not be easy – but it is still worth trying, and doing so should be the president’s highest priority at his talks with Xi Jinping this month.
No company – or country – wants to cause an AI-driven catastrophe. Despite what Trump may think, it’s the government’s job to make sure they can’t.
