MedChemBench separates interpreting molecular structures from choosing promising compounds. OpenAI says its scores do not establish that models can discover drugs or improve laboratory decisions.
MedChemBench is a 99-problem evaluation built from 117 papers and patents, testing whether models can reason from historical chemistry evidence across four distinct tasks. GPT-6.2 Astra earned 84.5% overall credit, but GPT-6 Astra’s 95.4% on molecular-structure tasks contrasts with much weaker performance on compound selection and numerical prediction. The results map specific strengths and gaps; they do not show that models can discover drugs or reliably guide laboratory decisions, and prior exposure to the source material cannot be ruled out.
01
The benchmark includes 761 molecular structures and asks models to reconstruct compounds, explain findings, select candidates, and predict properties.
02
On one 18-atom-ring task, GPT-6 Astra reconstructed the molecule correctly in all ten responses; GPT-5.6 Sol and GPT-5.6 Luna failed all ten times.
03
A PRMT5 example shows why candidate ranking is difficult: one compound blocked the protein more strongly, while another crossed a cell barrier better and degraded more slowly.
OpenAI introduced MedChemBench on November 20, a 99-problem test of whether AI can connect molecular structures with experimental evidence. Its results reveal an uneven skill set: GPT-6 Astra performed strongly at reconstructing molecules and explaining findings, but choosing compounds and predicting their properties remained harder—tasks central to deciding what scientists should test next.
The benchmark draws on 117 papers and patents and includes 761 distinct molecular structures. For most problems, models receive a snapshot of a research program: the biological target, researchers’ goals, compounds already studied and selected experimental findings. They must use that information to explain results, compare candidates or predict properties.
OpenAI grades four abilities separately: reconstructing molecular structures, explaining the science, ranking or selecting compounds, and predicting properties. Those checks distinguish getting the molecule right from making a useful judgment about it. Numerical predictions receive partial credit based on their closeness to measured values, rather than a simple right-or-wrong score.
OpenAI’s reported overall scores
84.5%GPT-6.2 Astra
At the xhigh reasoning setting, OpenAI reports GPT-6.2 Astra earned 84.5% of available credit.
83.3%GPT-6 Astra
GPT-6 Astra earned 83.3% at the same reasoning setting.
58.2%GPT-5.6 Sol
GPT-5.6 Sol earned 58.2% at the same reasoning setting.
Chemical drawings encode which atoms connect, how rings are formed and how atoms are arranged in space. Losing that information changes the compound being analyzed. One benchmark problem asks models to assemble a molecule from a central structure and two attached groups while preserving an 18-atom ring.
At xhigh, the highest reasoning setting shared by all eleven tested models, GPT-6 Astra reconstructed that molecule correctly in all ten responses. GPT-5.6 Sol and GPT-5.6 Luna failed all ten times. In selected answers OpenAI illustrated, Sol shortened the ring to 15 atoms, while Luna produced two disconnected fragments.
Across the structure tasks, GPT-6 Astra earned 95.4% of available credit. It earned only about half the credit for compound selection and numerical prediction. OpenAI cautions that these categories use different tasks and scoring rules, so the percentages are not directly equivalent measures of difficulty.
The prediction tasks address a recurring chemistry problem: improving one property can worsen another. A compound might act strongly against a disease-related protein yet be poorly absorbed or break down too quickly. Scientists must weigh several properties when deciding which related compounds to make and test.
One problem concerns inhibitors of PRMT5, a protein studied in cancer drug discovery. Given structures and earlier results, the model must predict three compounds’ ability to block the protein in cells, cross a cell barrier and resist breakdown by liver enzymes in a laboratory assay.
Later experiments, withheld from the model, split the advantages. Compound 23 blocked PRMT5 in cells at a lower concentration. Compound 24 crossed the barrier more readily and broke down more slowly in the enzyme assay. Predicting all three properties requires more than identifying the strongest blocker.
OpenAI’s evaluation used single-attempt responses without tools, feedback or new experiments. Prior model exposure to the published source material cannot be ruled out. The company explicitly says the results do not establish whether models can discover new drugs or improve laboratory decisions.
For now, MedChemBench measures reasoning from historical evidence. Its component scores identify specific weaknesses in choosing compounds and predicting measurements, rather than certifying a model to direct the next laboratory experiment.
Sources
openai.comCan AI help scientists identify promising drug candidates?
Reader comments
Newest comments first. Replies stay oldest first.