Hugging Face’s policy chief warns embedded AI evaluators need clearer standards

Putting outside evaluators inside AI labs leaves a harder question unresolved: what makes their assessments reliable? Hugging Face chief policy officer Irene Solaiman called for clearer evaluator standards and reproducible tests in remarks published by TechCrunch on October 19. She also argued that testing must cover complete AI agent systems, not just models.

Speaking during a discussion at TechCrunch Disrupt the previous week, Solaiman described evaluations as an evolving science. Different tests remain difficult to compare meaningfully, she said, and can lose their effectiveness relatively quickly. Her concern extends beyond whether a particular score is valid to whether the industry has enough qualified people and consistent methods to interpret it.

The push to put outsiders inside labs

The embedded-evaluator approach places outside reviewers within an AI company, giving them a closer view of its systems and safety work. On September 18, Accenture and Anthropic announced a partnership to establish such a team. Its remit includes probing models for failures, assessing whether they behave as intended, and testing safeguards.

TechCrunch reported criticism of Accenture’s qualifications for that role. In its partnership announcement, Accenture said it would draw on Faculty, the applied-AI business it had acquired. The company pointed to Faculty’s experience evaluating models for AI labs and building systems for government, defense and healthcare as evidence of its expertise.

Anthropic’s October 15 update described a more developed arrangement with METR, a nonprofit evaluator. Six METR employees had received company hardware, office space and internal access roughly matching Anthropic’s risk-assessment teams. Anthropic also said it was in early discussions with Hugging Face, alongside work already underway with SecureBio and Accenture.

That access has boundaries. Anthropic excludes categories including customer data, legally privileged information and model architecture details. It says METR can publish without its approval, but Anthropic can require redactions within predefined categories—and ultimately decides what falls within them. METR retains the right to make certain statements about the access and information it received.

Access is only part of the test

For Solaiman, a key test of trust is whether someone can reproduce an evaluation’s results. That requires sufficient access to the model itself. She highlighted the importance of open weights—the model parameters made available for others to use—in enabling that scrutiny. Her emphasis was on the ability to check results, rather than accepting a published score alone.

Anthropic’s own criteria for evaluators include expertise in frontier models and how evaluations can go wrong, independence, sufficient access and freedom to publish. Its definition of independence excludes holding equity in the company being assessed or being financially or operationally dependent on it. Those are the company’s stated requirements, not an industry-wide standard.

The next testing gap surrounds the model

Solaiman also challenged the model-only focus of testing. A harness—the surrounding tool used to access and operate a model—can substantially affect performance. A weaker model with a better harness could outperform a frontier model, she said. Evaluating the underlying model therefore does not, by itself, describe how the complete setup will perform.

That leaves a costly gap as evaluation expands to agents. Solaiman said tests of models are more plentiful than tests of harnesses, while better evaluations of complete agents are still being developed. She described agent evaluations as difficult and expensive. The unfinished work is measuring the full system people use, not simply adding another test of its model.