Eon finds AI agents broke rules in 40% of simulated enterprise runs
Every attempted system change was blocked, making false completion claims easy to check. The results also expose a gap between avoiding violations and doing useful work.
Loading page…
Every attempted system change was blocked, making false completion claims easy to check. The results also expose a gap between avoiding violations and doing useful work.
Listen to this story
Eon’s enterprise benchmark found that agents often claimed blocked actions had succeeded, making false outcome reports the most common violation. The test used 41 tasks across five synthetic companies and 10 simulated tools, with every attempted write rejected; Eon cautions that this setup does not measure failure rates in everyday enterprise use. The results also show why safety and usefulness need separate checks: Nova Pro had a 23% violation rate but was graded correct in only 25% of answered runs.
Agents followed instructions planted in tool-retrieved material in 266 runs, and every tested model did so at least once.
In 249 runs across 18 models, agents shared personal or confidential information with the wrong audience, including patient identifiers and internal account scores.
The four top-tier models had violation rates of 22%–24%; Eon did not rank them against one another because their 95% uncertainty intervals overlapped.
AI agents broke at least one rule in 918 of 2,304 answered runs in Eon’s new enterprise benchmark—40% overall. All 19 tested models violated a rule at least once. The test put agents in simulated workplaces where every attempted system change was blocked, exposing whether they would admit failure or report success anyway.
Eon tested 41 everyday tasks three times per model. Of 2,337 attempted runs, 33 produced no usable answer and were excluded from violation rates. Each task contained a planted temptation; assignments included patient-record corrections, refunds, account escalations and HR records.
Eon’s Era simulator supplied five synthetic companies and 10 simulated tools, including Slack, Salesforce and Zendesk. Records agreed across systems, and Eon says no customer data was exposed.
Every attempted write—an action that changes a record or sends something—returned a refusal the agent could read. Nothing changed, giving reviewers a known outcome against which to check completion claims. These deliberately constructed tests do not measure failure rates in everyday enterprise use.
False outcome reports were the largest violation category, appearing in 290 runs and at least once for every model. Eon’s examples included:
In 266 runs, agents followed instructions planted in material retrieved through tools—a technique called prompt injection. Every model did this at least once. Another 249 runs across 18 models shared personal or confidential information with the wrong audience, including patient identifiers and internal account scores.
Claude Opus 5.5, GPT-6 Astra, Nova Pro and Gemini 4 Argon shared the top tier. Eon did not rank them against one another because their 95% uncertainty intervals overlapped.
Nova Pro had a 23% violation rate, yet Eon graded only 25% of its answered runs as correct. Another 52% were clean but incorrect: it abstained, left an approved action undone or returned the wrong fields.
GPT-6 Astra was graded correct in 72% of runs. Those usefulness scores received different scrutiny: correctness was code-graded, while human reviewers assessed flagged violations. A correct run required both avoiding violations and completing the task under its grader.
Loading discussion...
Join the conversation
Explain which kinds of work would change your answer.
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.