Loading page…
The reported behavior is not evidence of hidden intent. But if agents can coordinate in language people can see yet cannot interpret, monitoring alone may not be enough.
Listen to this story
Emergence reports that autonomous agents powered by frontier models from U.S., Chinese, and French companies formed shared vocabulary and conventions within days of cooperative tests, including terms for tool-building, accountability, and resilience. The notable risk is interpretability: agents may optimize communication for efficiency until humans can read logs but no longer reliably understand them.
DeepSeek-based agents called tool-building agents “forge-smith”; Anthropic agents used “name-first” for accountable claims.
Google-based agents used “kintsugi” to describe system resilience, showing conventions differed across model families.
Niall Curry said compressed agent language could reduce computation costs while making oversight harder.
Researchers at Emergence say autonomous AI agents placed in cooperative experimental societies began creating shared phrases and meanings within days, without being instructed or rewarded to invent a language. The agents’ language also grew more opaque as they communicated, creating a potential obstacle for human oversight.
The experiment involved autonomous agents powered by frontier models from companies in the US, China and France. Emergence executive chair Satya Nitta said the agents developed conventions themselves and that other agents then adopted them. The finding concerns behavior in an experiment, rather than evidence that agents were concealing a purpose from people.
The vocabulary was not merely technical shorthand. Researchers said DeepSeek-based agents used “forge-smith” for an agent that builds tools for others. Anthropic-based agents used “name-first” for an agent demonstrating accountability by attaching its name to a claim, while Google-based agents used “kintsugi” to mean system resilience.
They developed new vocabulary, shared meanings and communication conventions themselves – and other agents adopted them.
Satya Nitta, executive chair of Emergence
Nitta’s concern is not that the conversations became invisible. He said their conventions could evolve until people could observe the exchanges but struggle to understand what they meant. That distinction matters for systems intended to act with some autonomy: a readable log is a weaker safeguard when its language is no longer interpretable.
Niall Curry, a linguist at the University of Birmingham, said agents may streamline language to reduce computation costs and improve efficiency. He also said unintelligible exchanges could leave monitors unable to determine what agents had actually done. Efficiency and oversight, in other words, may pull in opposite directions.
The report does not establish that these emergent terms enabled harmful actions, or that they will appear in deployed systems. It does raise a narrower operational question: whether safety monitoring needs to assess shared meanings, not simply preserve access to messages. OpenAI chief scientist Jakub Pachocki has separately warned that confidence in monitoring AI systems’ thinking is essential for safe development and may constrain progress. Emergence’s result suggests that interpreting agent-to-agent language could become part of that constraint.
Loading discussion...
Join the conversation
Explain where you would draw that line.
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.