Qwen3.8-Max Leads Open-Weight Commerce Agent Bench by Two Passes, With Audit Limits

Qwen3.8-Max holds a two-pass lead over DeepSeek V4 Pro in the published open-weight Commerce Agent Bench results, with 156 passes across three agent harnesses. The lead came from one harness, and the task-level records needed to fully reproduce the leaderboard are not publicly archived with immutable checksums.

Qwen3.8-Max completed 56 of 107 tasks through Accio, 47 through OpenClaw and 53 through Pi. DeepSeek V4 Pro recorded 54, 47 and 53 passes, respectively. The two-pass aggregate advantage therefore came entirely from Accio; the models tied in the other two harnesses.

It is not an overall lead. Claude Opus 5 led all evaluated models with 191 aggregate passes, 35 more than Qwen3.8-Max. Its best harness result was 66 of 107 tasks, leaving 41 tasks unfinished even at the published high-water mark.

Published open-weight results

01156 passes

Qwen3.8-Max

Qwen3.8-Max recorded 156 passes across the three published harness tables.

02154 passes

DeepSeek V4 Pro

DeepSeek V4 Pro recorded 154 passes in the same aggregate comparison.

Commerce Agent Bench contains 107 tasks spanning command-line, browser, file and document, and API or Model Context Protocol workflows. It covers product publishing, freight booking, storefront administration, payment operations, supplier analysis, web research and spreadsheet creation.

Each task starts in a fresh container within one of 14 offline software replicas. After an agent stops, the benchmark checks the resulting state of mock business services and any produced files. A task passes only if every required check passes, rather than because the agent described a plausible sequence of actions.

The materials identify several variables that can change a score: provider routing, model snapshots, prompt adapters, retry policies and judge endpoints. Qwen3.8-Max’s published rows also used a reasoning-content replay protocol, which carries its reasoning trace into later assistant messages and is described as necessary for comparable results.

The benchmark code is published under Apache 2.0 and its task data under CC BY 4.0, with commercial use permitted under those licenses. Yet Accio is reference-only and cannot be reproduced from a repository checkout, while OpenClaw is the shipped rerunnable harness. The published task-level result bundles are neither stored in Git nor available through immutable public archives with checksums, leaving full independent reproduction of the leaderboard unresolved.