Research survey calls for testing whole AI agents, not just their models

An AI agent can reach the right answer through very different paths—including shortcuts that make success a misleading signal. In Emerging Trends in AI Agent Evaluation, dated October 19, researchers argue that testing must cover the whole system, not just its model. Their survey maps 33 shifts, practices and open questions in how agents should be assessed.

The same outcome can hide different behavior

The paper contrasts a familiar testing template—ask a model a question and compare its response with a reference answer—with systems that plan, use external tools and act over many steps. Those agents depend on more than the language model supplying their responses.

The surrounding software, often called a scaffold or agent harness, is part of what needs testing. So are tools, testing environments and user interfaces. The authors argue that these components change what a result can establish: an agent's performance cannot automatically be treated as a measure of the underlying model alone.

That creates a problem when explaining improvements. A higher score may reflect a different model, but changes in tools, environments or the harness also need to be ruled out. Nor does using the same harness settle every comparison: the paper notes that it may interact differently with different models.

Long runs introduce another difficulty. Small differences early in a sequence can compound into sharply different outcomes, increasing variation across attempts. The authors also argue that real-world performance involves tradeoffs among cost, latency—the time taken to respond—and reliability. Accuracy alone does not capture those tradeoffs.

Doing the work versus helping a person

The survey is a map of evaluation methods and research questions, rather than a new ranking of agents. It organizes its 33 entries around how to evaluate, what to evaluate and who evaluates. Its authors list affiliations including Princeton, Stanford, the UK AI Security Institute and the Brookings Institution.

One important distinction concerns the purpose of deployment. The authors separate tests of whether an agent can complete work independently from tests of whether it can help a person work. They frame these as automation and augmentation: different questions that should not be collapsed into a single assessment of capability.

The proposed research agenda therefore stretches beyond isolated task completion. It includes:

• Human-agent interaction, studied through simulated users, lab experiments and field experiments.
• Real-world usage, domain-specific performance and economic impact.
• Dangerous capabilities, security, interaction harms and effects at the system level.

The evaluator is part of the result

The paper's final contrast is between benchmarks as neutral measurements and evaluations as the product of consequential choices. Evaluators decide on prompts, tools, budgets, environments, human intervention and statistical analysis. Those decisions shape the evidence that developers, policymakers and the public receive, the authors argue.

Its section on the evaluator ecosystem examines independence, funding, rising costs and public-sector involvement. It also considers influence in both directions: how AI labs shape evaluations, and how evaluations shape labs. These are presented as practical considerations for people contributing to evaluation, alongside the paper's methodological best practices and research directions.