Anthropic’s Amodei Pledges to Slow AI Releases as Needed for Safety at UN

Anthropic CEO Dario Amodei told the UN Security Council on Wednesday that his company would slow new AI releases as much as needed to keep them safe. He also urged global testing standards, but those remain proposals rather than shared rules. The reported remarks leave unclear what would trigger a delay.

A company pledge before the council

Amodei spoke by video at the September 23 meeting, held alongside the UN General Assembly. He said more powerful models would require stricter safety standards and better protection against misuse. If AI is managed poorly, he warned, it could pose a risk to humanity as a whole.

His promise concerned Anthropic’s pace of development and releases; it did not commit rival companies or governments to do the same. Amodei had urged a slowdown in a September 12 essay. At the council, he put Anthropic’s willingness to slow future releases in direct terms.

We will slow down as much as necessary in order to make sure that every successive AI technology that we release is actually safe.

Dario Amodei, Anthropic CEO, addressing the UN Security Council

Testing rules and public control

Amodei advocated global standards for testing new AI models and an international ban on using AI to develop biological weapons. The first proposal concerns how models would be examined; the second concerns a particular use. Neither proposal, as described, gives an outside body authority over Anthropic’s releases.

OpenAI chief Sam Altman pressed the question of who gets a say. He warned that AI could advance so fast people might no longer understand what was happening or intervene. Major decisions, he argued, should be shaped by democratic processes and accountable governments, not AI labs alone.

Altman called his approach a “middle path”: keep AI under human control while ensuring it benefits and empowers people. Amodei’s company pledge and Altman’s call for public decision-making address different parts of the same problem. Neither establishes a common release rule.

Resistance to international rules

French Foreign Minister Jean-Noël Barrot chaired the meeting and urged governments and companies to work together on rules. But a day earlier, President Donald Trump rejected international AI regulation as a “globalist scheme.” White House science and technology director Michael Kratsios said rapid AI progress was no reason to pause development or constrain it with new governance structures.

Another speaker showed why the debate is not simply about restricting AI tools. Hugging Face chief Clement Delangue said his company had been targeted by AI and had used AI to fend off attacks. He argued that open-source AI—technology others can access and adapt—could help defenders.