OpenAI has launched BioBasicsBench, a benchmark developed with Ligo to measure how AI agents interpret raw experimental data across 100 routine wet-lab tasks.
OpenAI has introduced BioBasicsBench, a benchmark designed to measure how well AI agents interpret raw experimental data across 100 routine wet-lab tasks. The benchmark was developed with Ligo, a biotech company focused on enzyme design.
BioBasicsBench includes 100 tasks across five areas of routine wet-lab work: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. Scientists created the tasks using data from their own experiments, pairing each question with expert-written grading criteria. Each task underwent an internal review by a Ligo scientist who had not written it, an external review by a PhD-level scientist working in a wet lab, and a final internal review.
The benchmark uses raw experimental evidence, including gel and blot images, plate photographs, microscopy stacks, sequencing reads, plate-reader exports and videos of bench work. Reference sheets, such as ladder charts and plasmid maps, support the interpretation of the experimental evidence.
OpenAI evaluated seven models on BioBasicsBench: GPT‑5.2, GPT‑5.4, GPT‑5.5, GPT‑5.6 Sol, GPT‑6.1 Sol, GPT‑6 Luna and GPT‑6 Astra. The evaluation covers 37 model and reasoning-effort settings, with five attempts on each of the 100 tasks per setting, for a total of 18,500 results.
At the ‘xhigh’ setting, which allows more reasoning per task, GPT‑6 Astra earned an average of 66.7% of the available credit, compared with 43.8% for GPT‑5.6 Sol.
| Model | Mean reward | 95% CI | Output tokens / task |
|---|---|---|---|
| GPT‑6 Astra | 66.7% | 60.3–73.1% | 4,704 |
| GPT‑6.1 Sol | 62.0% | 55.6–68.4% | 5,238 |
| GPT‑5.6 Sol | 43.8% | 37.6–50.0% | 9,099 |
| GPT‑6 Luna | 38.3% | 32.3–44.5% | 13,120 |
| GPT‑5.5 | 35.4% | 29.7–41.0% | 11,135 |
| GPT‑5.4 | 33.2% | 28.3–38.4% | 13,717 |
| GPT‑5.2 | 31.2% | 26.2–36.3% | 27,620 |
OpenAI reports that higher benchmark scores do not yet mean models can make reliable judgments in the lab. Examples of shortcomings include letting assumptions override controls, discounting data that contradicts a preferred interpretation, and miscounting colonies. In one colony-counting example, Astra’s counting marks resulted in an overcount of approximately 12% relative to the benchmark reference.
BioBasicsBench makes everyday wet-lab work a target for improvement, focusing on interpreting raw experimental data, checking controls and recognizing when a conclusion is not supported. OpenAI states that progress on these tasks could help scientists spend less time troubleshooting and more time deciding which experiments to pursue next. Tracking these capabilities will also help OpenAI understand their implications for biosafety and biosecurity.
Sources
This article was drafted with AI assistance and checked against the sources above. Company claims are reported as claims. Cover image is AI-generated.
The day's checked news, by email
One short email a day with the top stories. Free, and you can unsubscribe in one click.
Or follow for the latest news:
No comments yet. Start the discussion.
AI & SITrending
Three major investors say Anthropic's IPO valuation may need to fall from about $1.5 trillion to closer to $1 trillion after a sharp Wall Street sell-off.
Meta is rolling out Spaces, a feature that lets Muse users create custom app-like experiences inside the AI assistant by describing what they need, with updates from connected third-party apps.
