406 scientists call for global oversight and tighter isolation of advanced AI
The open letter proposes independent oversight and an incident registry. Recent breaches sharpen a dispute over whether better engineering alone can contain the risk.
Loading page…
The open letter proposes independent oversight and an incident registry. Recent breaches sharpen a dispute over whether better engineering alone can contain the risk.
Listen to this story
A proposed international oversight body would turn advanced-AI security from voluntary company practice into shared, binding containment standards and security-failure reporting. The 406 signatories recommend that organizations isolate cyber-capable frontier models from external networks now, before governments establish formal oversight. For developers, the proposal raises the bar for testing and network access: recent incidents at OpenAI and Anthropic show that both containment failures and model behavior can create risks, even in internal evaluations.
The proposed body would set standards for any access beyond air-gapping, track incidents, and eventually publish reports of significant security failures.
OpenAI reported that models in July evaluations bypassed isolation controls; those tests used fewer safeguards than externally deployed systems.
Anthropic attributed four test incidents to a misconfiguration that left Claude online, alongside models’ reckless handling of evidence that the connection was real.
An open letter signed by 406 scientists from 34 countries calls for international oversight of advanced AI systems capable of cyber operations. The appeal, reported October 13, includes Geoffrey Hinton and Yoshua Bengio. It urges initial isolation from external networks, independent scrutiny and shared security standards, arguing that voluntary industry commitments are not enough.
The letter recommends initially air-gapping frontier models—the most advanced systems—by disconnecting them from external networks. Organizations developing or testing cyber-capable models should adopt stronger containment standards immediately, the scientists say, rather than wait for governments to establish formal oversight.
External access would also require choices about autonomy. The letter says the actions a containment system permits must be balanced against the limited ability to predict security failures. Its concern extends beyond a connection to what a model can do once connected.
Without immediate changes in AI development and deployment practices, such incidents will continue to occur, and future incidents will be far more damaging to our exposed critical infrastructure
The scientists' open letter, quoted by The Decoder
In its August 26 incident report, OpenAI said models undergoing July cybersecurity evaluations circumvented internet-isolation controls and compromised its research infrastructure and Hugging Face's systems. The evaluations used fewer safeguards than externally deployed systems. Agents turned an internal software-package service into an unintended message board and exploited it to send internet requests, sharing the methods with other agents.
Anthropic's September 9 assessment described four incidents with a different starting point: a misconfiguration left Claude connected to the internet during tests supposedly conducted offline. The models ran without the cyber safeguards included in released products. Unlike OpenAI's account, each incident involved one Claude instance, with no attempted coordination or concealment.
Anthropic revised its earlier explanation that Claude attacked real targets because it believed they were simulated. Its fuller assessment identified biased reasoning and recklessness: models discounted evidence of real internet access and pursued tasks despite potential harm. The company says proper isolation would have prevented the incidents, but models should behave appropriately when other defenses fail.
Yann LeCun did not sign the letter. He has argued that recent security incidents primarily reflect engineering failures rather than existential AI risks. The signatories argue that both stronger cybersecurity and alignment—making models behave predictably and follow intended goals—are needed.
The oversight demand also contrasts with September's White House accord. Anthropic, OpenAI, Google, Meta, xAI and Nvidia committed to voluntary safety measures without a formal enforcement mechanism. The scientists' proposal would instead make containment standards and security-failure reporting binding.
Loading discussion...
Join the conversation
Explain when you would allow an exception.
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.