OpenAI chief scientist proposes shared AI development limits tied to prediction scores
Jakub Pachocki disclosed internal training measurements to support his proposal. But the scores are not directly comparable across labs and miss some routes to stronger AI.
The proposal would make frontier-model development contingent on agreed safety milestones: labs would define capability boundaries, test safety claims adversarially, and hold back progress until monitoring improves. The framework is voluntary, and its prediction scores are only a possible shared signal—not a direct measure of task ability or a comparable yardstick across labs today. Pachocki argues the stakes could rise as AI begins accelerating pretraining research as soon as 2027.
01
On OpenAI’s fixed internal code evaluation set, Pachocki disclosed log-perplexity scores of about 0.99 for GPT-3.5 and 0.91 for GPT-4.
02
He cited a model near 0.76 log-perplexity that produced what he called groundbreaking results on open mathematics problems.
03
Different datasets and training mixtures make cross-lab score comparisons difficult; chain-of-thought reasoning can also boost capabilities without lowering perplexity.
OpenAI Chief Scientist Jakub Pachocki wants frontier AI developers to agree on limits before their models acquire capabilities they cannot reliably anticipate. In a November 22, 2026 post, he proposed grounding those limits partly in perplexity—a measure of prediction quality—and disclosed selected OpenAI scores to encourage shared safety standards.
Developers would agree on evaluation methods, adversarial testing of safety arguments, and transparency standards. Development limits would remain until defined improvements in understanding and monitoring AI behavior were achieved. These are proposed industry commitments, not adopted rules. Pachocki acknowledged that disclosing internal scores gives competitors useful information, but argued that shared safety measures outweigh that concern.
The urgency comes from his expectation that AI could accelerate research on pretraining algorithms as soon as 2027. He extrapolates from a model at roughly 0.76 log-perplexity that he said produced groundbreaking results on open mathematics problems. He expects AI to help safety research too, but doubts that progress will keep pace with the full potential acceleration in pretraining.
The measurements behind the proposal
Perplexity measures how well a model predicts the next token, a unit of text, on average. Lower scores mean better predictions. OpenAI generally tracks its logarithm, also called cross-entropy loss. The disclosed figures use a fixed snapshot of evaluation data drawn from OpenAI’s internal codebase, kept separate from training.
OpenAI’s disclosed log-perplexity measurements
≈0.99GPT-3.5
Pachocki’s reported score on OpenAI’s main evaluation dataset.
≈0.91GPT-4
Pachocki’s reported score on the same dataset.
Pachocki argues that pretraining—the initial learning process across a large dataset—has repeatedly produced unexpected abilities as prediction improves. Conventional task benchmarks can become saturated, he notes, while he believes pretraining measurements can remain informative across model generations.
A shared score still has gaps
Scores are not automatically comparable. Labs use different evaluation datasets and training mixtures. Pachocki argues that greater transparency could help establish comparisons.
Prediction does not directly measure a useful task. Its proposed value is signaling abilities that may emerge, not identifying exactly what those abilities will be.
It misses other routes to stronger performance. Chain-of-thought reasoning—generating intermediate reasoning text—can sharply improve capabilities without reducing the model’s perplexity.
Pachocki proposes using perplexity alongside other measurements, not alone. His suggested condition for crossing a development limit is a concrete monitoring advance, such as a successor to chain-of-thought monitors. The proposed bargain is a sequence: agree on a boundary, specify the safety improvement needed, then permit further development.
Reader comments
Newest comments first. Replies stay oldest first.