Anthropic’s Fable 5.2 passes 41.3% of freelance projects in Scale’s benchmark

AI agents have cleared a much larger share of a test built around finished freelance work. Anthropic’s Fable 5.2 now leads Scale’s Remote Labor Index with a 41.3% automation rate. That means human reviewers judged its output acceptable against professional reference work—not that AI has taken over that share of freelance jobs.

The lead is substantial. Fable 5.2 completes nearly twice the share of projects as the next-best listed system, Opus 5.5. Its predecessor, Fable 5.1, scores 17.92%, while Fable 5 scores 15.83%. These are results for completed assignments that meet the benchmark’s quality bar, rather than points earned for answering isolated questions or finishing individual steps.

The historical contrast is even larger, but it needs the right baseline. When Scale AI and the Center for AI Safety introduced the benchmark on October 28, 2025, its best automation rate was 2.5%. The leaderboard identifies that early leader as Manus. The rise to 41.3% therefore tracks progress at the top of the benchmark, not an improvement from an earlier Fable score of 2.5%.

41.30%Fable 5.2

The highest automation rate on Scale’s published leaderboard.

21.25%Opus 5.5

The second-highest listed automation rate.

20.83%GPT 6 Astra

The third-highest listed automation rate.

A finished file, not a plausible answer

RLI’s dataset contains 240 projects collected from 358 verified Upwork freelancers across 23 domains. Researchers filtered an initial pool of 550 projects for completeness, reproducibility and professional quality. Each retained assignment includes a brief, input files, the accepted human deliverable, and reported time and cost. Human completion time averages 28.9 hours, making these more substantial assignments than a quick prompt-and-response exercise.

Trained reviewers compare an agent’s submission directly with the professional reference. A project passes only if it satisfies the brief at least as well as that reference and would be acceptable to a reasonable client. The review covers complex files, including video, 3D models and computer-aided design files. Scale reports 94.4% reviewer agreement for the automation metric, using majority votes from three independent evaluators.

The agents do not work from text alone. Scale’s published methodology provides common prompts, execution environments and specialized tools for generating images, speech and video. It describes both command-line and mouse-and-keyboard environments, with a maximum generation budget of $30 per task. The leaderboard records each agent’s best performance across environments, so the score describes a tool-assisted system rather than a model answering without outside tools.

Official scores use 230 private projects; only 10 projects and the evaluation platform are public. Scale says that split reduces the risk of models encountering test material during training. The leaderboard does not provide confidence intervals for automation rates, leaving readers without a displayed statistical uncertainty range around each score.

The gains are uneven—and the test has boundaries

The Decoder reports that Scale Labs highlighted Fable 5.2’s largest gains in audio and music editing, architecture and interior design, and 3D modeling. According to that account, nearly 50 successful completions were projects no previously tested agent had solved. Scale also identified industrial product design and animation as areas where the system still struggles. Those domain-level details come from The Decoder’s account of Scale’s statements.

RLI also deliberately leaves out several kinds of work. Its exclusions include:

• Direct client interaction, such as customer service or tutoring.
• Physical labor, which falls outside the remote-work tasks tested.
• Long-term evaluation, such as assessing search-engine optimization results.

Even within that narrower scope, most projects remain below the acceptance threshold. Scale’s general failure analysis identifies amateur-quality work, missing or incomplete deliverables, broken file formats and inconsistencies across files. Those categories describe the benchmark’s broader failure patterns, not a fresh breakdown of Fable 5.2’s unsuccessful submissions. The new score establishes a higher completion rate, while leaving substantial room between benchmark leadership and dependable delivery.