GLM-5.3’s Cheap Retries Put Fable 5’s Coding Premium Under Pressure
Together AI’s benchmark run suggests first-attempt parity is not enough to justify Fable 5’s price for general coding. But a Rust and serialization advantage, plus benchmark-specific cost assumptions, leave room for targeted routing.
Listen to this story
The audio brief
Story brief
3 key pointsTogether AI's DeepSWE evaluation suggests retry economics can outweigh a near-tied first-shot score: GLM-5.3 reached 87.6% pass@4 versus Claude Fable 5's 84.1%, while reported rollout costs were $3.99 and $21.63, respectively. A GLM-first cascade was estimated at 81.1% accuracy for $10.74 per task, but this is not a clean production benchmark: GLM's cost used index-only accounting, Fable had more infrastructure errors, and routing assumptions included verifier-based escalation and conditional independence.
- 01
At pass@1, Fable led 69.7%-69.0%; Together AI considered the 0.7-point gap within the models’ error bands.
- 02
GLM won five of eight reported domains, including JavaScript (90% versus 75%) and concurrency and durability (62% versus 45%).
- 03
Fable led Rust (85% versus 70%) and data modeling and serialization (88% versus 79%), favoring specialty-based routing.
Claude Fable 5 narrowly led GLM-5.3 on the first coding attempt in Together AI’s DeepSWE run. Once retries entered the picture, GLM pulled ahead while carrying a reported $3.99 rollout cost, against $21.63 for Fable. The result shifts the practical choice from a single benchmark score to the cost of reaching a passing patch.
The first patch is not the decision
The comparison tested both models at maximum effort on 113 DeepSWE software-engineering tasks, with four trials per model and 904 rollouts in total. Fable reached 69.7% pass@1 under official scoring, compared with 69.0% for GLM; Together AI characterized the 0.7-point difference as within the models’ error bands.
Pass@k measures whether at least one of several attempts passes a task’s hidden test suite. GLM’s advantage grew at higher attempt counts: 81.1% versus 77.1% at pass@2, and 87.6% versus 84.1% at pass@4. Its reported return was 17 solves per $100, versus three for Fable.
Same first-shot range, sharply different stated cost
GLM-5.3
GLM-5.3 posted 69.0% pass@1 at a reported average cost of $3.99 per rollout.
Claude Fable 5
Claude Fable 5 posted 69.7% pass@1 at a reported average cost of $21.63 per rollout.
Specialties, not a broad portfolio
The aggregate scores conceal distinct task strengths. GLM led on JavaScript, 90% to 75%, and on concurrency and durability, 62% to 45%. Fable led on Rust, 85% to 70%, and data modeling and serialization, 88% to 79%. GLM won five of the eight reported task domains; Fable won three.
The routing case is narrow
- The models had a 0.65 per-task correlation and together solved 106 of 113 tasks. Their overlap limits the coverage benefit of routinely running both.
- Together AI calculated that a GLM-first cascade, escalating after verifier rejection, reached 81.1% accuracy at $10.74 per task. That calculation assumes a verifier decides escalation and conditional independence given the task.
The benchmark has operational boundaries
The official scores excluded infrastructure errors. Fable had 16 such errors and GLM had one; counting them as failures put Fable at 67.3% pass@1 and GLM at 68.8%. Fable’s cost matched recorded trajectory totals, while GLM’s was index-only because per-turn trajectory files for its batch were unavailable publicly at analysis time.
Sources
- together.aiGLM-5.3 vs. Claude Fable 5 on DeepSWE: Cost, Coding, and Routing
Reader comments
Newest comments first. Replies stay oldest first.