Claude Opus 5.5 leads GDPx’s AI benchmark across 11 specialized domains
The evaluation draws on 14 specialized data partners and includes cost comparisons. Its overall ranking is only one view of the results.
Loading page…
The evaluation draws on 14 specialized data partners and includes cost comparisons. Its overall ranking is only one view of the results.
Listen to this story
GDPx v1.0 brings 206 tasks across 11 specialized domains into one benchmark, with field-by-field comparisons alongside an overall ranking. Claude Opus 5.5 leads the published results, but GDPx scores are not an independently verified assessment, and its domain table displays only 13 of 20 models. A cost chart adds resource context, yet excludes Muse Wonder Full because it has no public price, meaning the performance and cost views cover different model sets.
The displayed overall scores are Opus 5.5 at 12.0, Gemini 4 Carbon at 10.9, and GPT-6 Astra at 9.6.
GDPx lists 14 specialized data partners, with some domains drawing multiple contributors.
Chip design has three listed partners; materials science has two.
Claude Opus 5.5 takes first place in GDPx v1.0’s displayed results, scoring 12.0 ahead of Gemini 4 Carbon and GPT-6 Astra. The benchmark measures frontier AI models across 206 tasks in 11 domains, bringing fields such as chip design, biology and legal work into a single evaluation.
Gemini 4 Carbon scores 10.9, followed by GPT-6 Astra at 9.6. Muse Wonder Full is next at 9.0, ahead of Claude Fable 5.1 at 8.5 and GPT 6.1 Sol at 8.4. These are the benchmark’s published GDPx scores, rather than an independently verified assessment of the models.
GDPx lists 14 specialized data partners. Some domains have several contributors: chip design has three, while materials science has two. The coverage also reaches cybersecurity, finance, quantitative trading, hardware design, alignment, software engineering and recursive self-improvement. Those sit alongside biology and legal work in the benchmark’s 11-domain comparison.
GDPx reports 206 tasks in the evaluation.
The benchmark spans 11 domains.
The page identifies 14 specialized data partners.
The domain-comparison view displays 13 of 20 models, with all 11 domains selected. It lets readers compare models by field rather than relying solely on the overall score. The displayed table ranges from Opus 5.5’s 12.0 to GLM 5.3’s 2.7; it is not the full 20-model selection shown elsewhere on the page.
A separate cost-versus-performance chart plots GDPx score against average cost per task. Its cost axis uses a logarithmic scale, and the interface also offers output tokens as an axis option. That gives the page a resource-use comparison alongside its ranking by score.
One model illustrates the difference between those views. Muse Wonder Full appears in the score table, but GDPx excludes it from the cost chart because it has no public price. Its performance result is therefore visible without a corresponding public cost comparison.
Story updates
SemiAnalysis is offering 168 transcripts of AI systems tackling specialized work, from debugging training runs to designing chips and preparing financial documents. Each partner has open-sourced records for one task, while withholding the rest to prevent hill-climbing. The release lets readers inspect selected attempts, not just the percentages attached to them.
SemiAnalysis is offering 168 transcripts of AI systems tackling specialized work, from debugging training runs to designing chips and preparing financial documents. Each partner has open-sourced records for one task, while withholding the rest to prevent hill-climbing. The release lets readers inspect selected attempts, not just the percentages attached to them.
The rollout page presents task descriptions alongside excerpts of the systems’ messages and tool use. These records show actions such as listing workspace files, reading instructions, checking installed software and encountering errors. A download offers the full collection of 168 transcripts; individual task entries also provide a Watch link.
SemiAnalysis calls these records rollouts. Its disclosure policy draws a clear boundary: each partner contributes the rollouts for one task, rather than opening all of its evaluation records. The stated reason for keeping the others private is to prevent hill-climbing. The public collection is therefore a selected window into the evaluation, not its complete record.
The rest remain private to prevent hill climbing.
The displayed results vary sharply with the assignment. For Extrapolate RSI’s training-run audit, the system must investigate why a large reinforcement-learning run stopped improving, then find and prove the bugs. The entry lists 12 runs and names Claude Opus 5.5 as best at 40%.
Aria’s chip-design task asks for a chip that runs TensorFlow Lite int8 networks, starting with a fixed pin list and ending with a signed-off layout. Speed, size and efficiency are part of the assignment. Its displayed best result is also Claude Opus 5.5, at 39%.
Other entries reach the top of the displayed scale. Arithmetic’s cybersecurity assignment asks systems to find and prove security flaws in a live enterprise environment built around a secrets vault. Third Moment’s trading assignment asks them to beat a reference Apple-stock strategy on hidden, out-of-sample trading days. The page lists these task-specific best results:
Cybersecurity: Gemini 4 Argon, 100%, across an entry listing 12 runs.
Quantitative trading: Claude Opus 5.5, 100%, across an entry listing 12 runs.
The business-document tasks require concrete deliverables. Living Context’s finance assignment starts with a colleague’s half-finished workbook and asks for a completed quality-of-earnings databook for a buyout target. Chakra Legal’s task requires redlining a startup’s Series B charter for the lead investor, using the signed term sheet, capitalization table and client instructions.
SemiAnalysis lists Claude Opus 5.5 as the best model on both entries, but the displayed figures are 0% for the finance workbook and 44% for the legal redline. Each entry lists 12 runs. Those figures belong to the named assignments, rather than a general score for finance or legal work.
Physical-design and scientific assignments add different finish lines. Beautiful Machines asks for a compact compressor’s injection-molded housing in SolidWorks, including drawings and a manufacturing plan; GPT-6 Astra is listed as best at 9%. ValenceForge asks systems to operate a transmission electron microscope through a measurement campaign and support every scientific result with evidence. That entry lists GPT-6 Astra as best at 63%.
Loading discussion...
Join the conversation
Explain whether performance alone earns a place on the leaderboard.
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.