Mercor and Google release 85-task benchmark for AI engineering designs

Mercor and Google introduced Physical-Engineering’s Last Exam on November 3, a benchmark that asks AI agents to produce engineering designs, not answer textbook questions. It spans 85 tasks across four disciplines, with 29 tasks released publicly. Its grading puts safety and feasibility ahead of cost savings—a stricter finish line than getting most calculations right.

A project folder, not a textbook question

PLE gives agents the kind of messy starting point engineers inherit on a project. Each task runs inside an isolated project folder containing about 61 files. Those include PDFs, spreadsheets, emails, two-dimensional drawings and three-dimensional building models. Outdated revisions are deliberately included as traps, making document selection part of the assignment rather than a preliminary step someone else has already completed.

Agents have no internet access and up to six hours to identify the governing codes and physical limits, perform calculations and produce a deliverable. The intended standard is work a licensed reviewer would accept. The benchmark covers four distinct kinds of engineering work:

• Mechanical: 30 tasks, including designing a cooling loop and arranging equipment in a three-dimensional hall without clashes.
• Electrical: 27 tasks, including sizing a switchboard and producing its cable schedule.
• Civil: 18 tasks, including laying out a campus with grading and utilities.
• Structural: 10 tasks, including designing a roof deck.

The tasks came through two routes. Engineers in Mercor’s network built 44 end to end, including project folders, prompts and grading rules. Google developed another 41 through a process in which a Creator agent constructs a task and a Solver agent tries to break it. Engineers validated tasks from both routes.

Safety checks come before savings

PLE awards points for cost efficiency only after a design clears every hard safety and feasibility check. A cheaper design cannot earn that credit by overlooking a mandatory requirement. Mercor frames this as the difference between an output that looks largely correct and one suitable for physical engineering, where designs must obey physics and building codes.

That distinction changes how the results read. Mercor says the top agent met most individual constraints on average, yet fewer than half its designs passed every hard check. The two measurements answer different questions: how many requirements were satisfied, and whether a design cleared the full set of mandatory checks.

91%Individual constraints met

Mercor’s reported average for the top agent.

41%Designs clearing every hard check

Mercor’s reported share for the top agent’s designs.

Two published top scores

Mercor’s November 3 announcement reports testing 14 frontier models across seven agent harnesses—the software that directs a model’s actions. It names Claude Opus 5.5 running in Claude Code as the leader on the hidden test set at 48.3%, followed by GPT 6 Astra at 45.8% and Gemini 4 Argon at 40.6%.

There is an unresolved numerical discrepancy. The research paper, first submitted October 21 and revised October 30, reports a strongest-configuration mean score of 50.6%. Neither source explains the difference, so that figure and the blog’s 48.3% hidden-test result should not be treated as interchangeable. Both give average resource use of 27 million tokens and 63 minutes per task for their strongest configuration.

Mercor also reports that model choice mattered more than harness choice: scores across models spanned up to 50 points, while switching harnesses for the same model produced a median change of about three points.

For developers looking to participate, Google has released the public tasks, grading tools and evaluation container. The competition runs against a hidden test set. Mercor says 13 Google-sponsored models can be run for free, while prizes will be announced later.