OpenAI introduces 100-task lab benchmark as AI still misreads experimental evidence

AI agents are getting better at reading lab evidence, but can still choose the wrong experimental samples after correctly spotting visual differences. OpenAI introduced BioBasicsBench on November 6, a 100-task evaluation developed with biotech company Ligo. Its results show stronger performance from GPT-6 Astra alongside errors that could send subsequent lab work in the wrong direction.

Raw evidence, not polished figures

The benchmark targets routine analysis in wet labs, where scientists experiment with biological samples. OpenAI argues that tests built around images and charts selected for scientific papers can miss the ambiguity of everyday experiments: failed runs, conflicting observations and images that require troubleshooting rather than a confident answer.

Tasks use evidence from scientists’ own experiments, including gel images, colony plates, microscopy, sequencing files, instrument exports and videos of bench work. They span five areas, from protein methods and imaging to molecular cloning. Requests range from reading a DNA sample’s molecular weight to interpreting a detailed assay.

Scientists paired each question with expert-written grading criteria. Every task passed through review by another Ligo scientist, an external PhD-level wet-lab scientist and a final internal reviewer. Reviewers examined the evidence, question, rubric, author notes and a graded model attempt.

What the scores actually measure

OpenAI evaluated seven of its models across 37 combinations of model and reasoning effort. Each setting received five attempts at every task, producing 18,500 results. At the xhigh setting, which allows more reasoning per task, Astra earned substantially more credit than GPT-5.6 Sol.

Available credit earned at xhigh effort

0143.8%

GPT-5.6 Sol

OpenAI reports a 95% confidence interval of 37.6–50.0%.

0266.7%

GPT-6 Astra

OpenAI reports a 95% confidence interval of 60.3–73.1%.

Those percentages are average shares of available credit, not the proportion of experiments handled correctly. Answers earn points against individual criteria covering conclusions, measurements, reasoning or recognition that evidence is insufficient. That lets the evaluation distinguish a correct conclusion from an analysis that is only partly right.

Seeing the control—and following it

Some improvements came from noticing small but decisive details. In one cloning task, Sol treated a colony-free plate as a useful negative control—a check on whether the experiment behaved as expected. Astra spotted that its label named the wrong antibiotic and rejected the plate as a reliable control.

Astra also better recognized when an image could not settle a question. In a cell-imaging task, Sol interpreted brightness from a dye as evidence of cell death. Astra noticed that another dye could interfere with the signal and called for better controls to distinguish them.

But OpenAI cautions that higher scores do not yet mean reliable lab judgments. In another cloning task, Astra assumed that a larger DNA product indicated success, overriding the comparison supported by the controls. Astra identified the visual groups correctly but reversed what they meant. Its selected colonies would have led to purification of the wrong plasmid, a DNA molecule used in the experiment.

Other failures involved evidence Astra had already inspected. It described compartments inside cells as generally darker while discounting lighter examples in the same image, despite noticing that focus changed their appearance. On a colony-counting task, it missed visible colonies, failed to separate close neighbors and marked empty agar. The resulting count was approximately 12% above the benchmark reference.