Superpower Daily
Anthropic Says Claude Found Fixes Across 10 Alignment Failures,...TeamPCP’s Trivy Attack Hit 2,500 Organizations; AI Tool Backdoo...Meta’s EvoHarness-RL Takes Qwen3-8B to 96.9% on ALFWorld, Near...Decathlon Deploys Chronos-2, Cuts 12-Week WAPE by 11–15 PointsSalesforce Says Agentforce Passed $1.5B ARR as It Pushes Consum...JAMA Paper Says Autonomous AI Could Surpass Physicians and AI-A...Anthropic Says Claude Found Fixes Across 10 Alignment Failures,...TeamPCP’s Trivy Attack Hit 2,500 Organizations; AI Tool Backdoo...Meta’s EvoHarness-RL Takes Qwen3-8B to 96.9% on ALFWorld, Near...Decathlon Deploys Chronos-2, Cuts 12-Week WAPE by 11–15 PointsSalesforce Says Agentforce Passed $1.5B ARR as It Pushes Consum...JAMA Paper Says Autonomous AI Could Surpass Physicians and AI-A...
Toolspublished

Meta’s EvoHarness-RL Takes Qwen3-8B to 96.9% on ALFWorld, Near Claude Opus 4.5

The result is a strong benchmark showing for learned state management, not yet evidence that the framework improves the long-running enterprise workflows it is designed to target.

By 3 min read
Meta’s EvoHarness-RL Takes Qwen3-8B to 96.9% on ALFWorld, Near Claude Opus 4.5
Sign in to save and react

Listen to this story

The audio brief

About 1:34
0:001:34
Read transcript
Meta AI and University of Illinois researchers say they trained Qwen3-8B to manage its own external state, pushing it to a 96.9 percent success rate on ALFWorld. That slightly tops the reported 96.4 percent for Claude Opus 4.5—and comes in far above Qwen3-8B’s 47.9 percent ReAct baseline. The important detail is what was trained. EvoHarness-RL makes the agent’s harness—the runtime layer that tracks state, feeds back execution results, and controls workflow—part of the learned policy. Its workspace divides external information into three buckets: Belief, the agent’s current view of the environment; Progress, its completed and pending subgoals; and Experience, reusable knowledge from earlier tasks. Four actions let it track, commit, recall, and note information. The training happens in two steps. First, supervised fine-tuning teaches the model to turn messy interaction logs into structured state. Then cost-aware reinforcement learning teaches it to judge whether retrieving or updating that state is worth the extra tokens and compute. The result also beats reported SkillRL and SkillOS scores of 89.9 and 80.2 percent. Prompt-time BPE improved GPT-4.1 by 22.1 points and GPT-5 by 25.7. But every result comes from the same text-based benchmark. The real deployment test is whether this learned memory policy survives enterprise workflows where state is incomplete, constantly changing, and expensive to consolidate.

Story brief

3 key points

Researchers from Meta AI and the University of Illinois Urbana–Champaign report that training Qwen3-8B to manage its own external state lifted ALFWorld success to 96.9%, versus 47.9% for its ReAct baseline and a reported 96.4% for Claude Opus 4.5. EvoHarness-RL makes memory operations part of the learned policy and prices them against token and compute costs. The result strengthens the case for adaptive agent...

  1. 01

    EvoHarness-RL organizes external state into Belief, Progress, and Experience, operated through track, commit, recall, and note actions.

  2. 02

    Cost-aware reinforcement learning teaches the agent when memory access or updates justify their token and compute expense.

  3. 03

    The trained Qwen3-8B beat reported SkillRL and SkillOS results of 89.9% and 80.2% on ALFWorld.

Meta AI and the University of Illinois Urbana–Champaign say their EvoHarness-RL framework let Qwen3-8B reach 96.9% average success on ALFWorld, edging Claude Opus 4.5’s reported 96.4% score. The striking comparison is bounded: it comes from a text-based benchmark of multi-step tasks, while the framework’s larger pitch is for agents working across long, changing workflows.

The runtime layer becomes part of the training target

An agent’s harness is the runtime layer that supplies execution feedback, state tracking and control flow around the model. EvoHarness-RL treats management of that layer as a learned behavior rather than a fixed set of developer-written instructions. The approach asks the model to decide when it should consult, change or consolidate the information outside its context window.

Its unified workspace separates external state into Belief, meaning the agent’s current view of its environment; Progress, meaning completed and pending subgoals; and Experience, meaning knowledge carried across tasks. The model operates those areas through four meta-actions: track, commit, recall and note.

Diagram of the Belief, Progress and Experience workspace used by EvoHarness-RL.
The BPE interface groups the agent’s external state into belief, task progress and reusable experience. Source: venturebeat.com.

Tool use has a budget

The training recipe has two stages. Supervised harness fine-tuning teaches the base model to turn messy interaction logs into structured external state. Cost-aware reinforcement learning then teaches it to weigh the benefit of retrieving or updating that state against the added token and compute cost.

The reported ALFWorld comparison
96.9%EvoHarness-RL-trained Qwen3-8B

The trained Qwen3-8B scored 96.9% average success on ALFWorld, 49.0 percentage points above its baseline ReAct counterpart.

96.4%Claude Opus 4.5

Claude Opus 4.5 scored 96.4% out of the box in the reported ALFWorld comparison.

A benchmark gain, with a broader but unproven deployment case

The Qwen3-8B result also exceeded the reported scores for SkillRL, at 89.9%, and SkillOS, at 80.2%. Researchers further found that adding the BPE harness at prompt time improved GPT-4.1 by 22.1 percentage points and GPT-5 by 25.7 points. Those figures suggest the workspace can help models beyond the one trained with EvoHarness-RL, but they remain results from the same reported evaluation.

The researchers describe two behaviors that explain the intended efficiency gain. During harness annealing, an agent reduces external-state queries as routine patterns are absorbed during training. During harness evolution, it increases use of Belief and Experience when it encounters a novel environment or a difficult situation.

Where the framework is meant to earn its keep

For short, stable tasks, the researchers say ReAct or standard retrieval-augmented generation may already be sufficient. They position BPE as more useful when an agent operates for hours, days or weeks and needs to keep an updated, compressed record of its environment, unfinished work and prior failures.

The implementation is designed to sit above domain-specific tools through an environment adapter, rather than require a wholesale replacement of an existing agent framework. The unresolved test is operational: whether the benchmark’s learned balance between memory access and action holds up when enterprise state is incomplete, changing and costly to consolidate.

Sources

  1. venturebeat.comMeta AI's EvoHarness-RL boosts agent efficiency | VentureBeat

Related coverage

Continue reading