OpenAI has introduced BioBasicsBench, a benchmark measuring how AI agents interpret raw experimental data across 100 routine wet-lab tasks, developed with Ligo.
By Hermes-Vector AI Desk
OpenAI has introduced BioBasicsBench, a benchmark designed to measure how well AI agents interpret raw experimental data across 100 routine wet-lab tasks. The benchmark was developed with Ligo, a biotech company focused on enzyme design.
BioBasicsBench tests whether agents can interpret raw experimental data, identify errors, and recognize when the evidence cannot support a conclusion. The benchmark uses raw experimental evidence, including gel images, colony plates, microscopy, and instrument readouts.
The 100 tasks span five areas of routine wet-lab work: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. Tasks range from simple requests, such as reading the molecular weight of a DNA sample in a gel, to detailed assay analysis.
Scientists created the tasks using data from their own experiments, pairing each question with expert-written grading criteria. Each task underwent an internal review by a Ligo scientist who had not written it, an external review by a PhD-level scientist working in a wet lab, and a final internal review.
OpenAI evaluated seven models on BioBasicsBench: GPT‑5.2, GPT‑5.4, GPT‑5.5, GPT‑5.6 Sol, GPT‑6.1 Sol, GPT‑6 Luna and GPT‑6 Astra. The evaluation covers 37 model and reasoning-effort settings, with five attempts on each of the 100 tasks per setting, for a total of 18,500 results.
At the ‘xhigh’ setting, which allows more reasoning per task, GPT‑6 Astra earned an average of 66.7% of the available credit, compared with 43.8% for GPT‑5.6 Sol.
Performance at xhigh reasoning effort:
| Model | Mean reward | 95% CI | Output tokens / task |
|---|---|---|---|
| GPT‑6 Astra | 66.7% | 60.3–73.1% | 4,704 |
| GPT‑6.1 Sol | 62.0% | 55.6–68.4% | 5,238 |
| GPT‑5.6 Sol | 43.8% | 37.6–50.0% | 9,099 |
| GPT‑6 Luna | 38.3% | 32.3–44.5% | 13,120 |
| GPT‑5.5 | 35.4% | 29.7–41.0% | 11,135 |
| GPT‑5.4 | 33.2% | 28.3–38.4% | 13,717 |
| GPT‑5.2 | 31.2% | 26.2–36.3% | 27,620 |
OpenAI reports that higher benchmark scores do not yet mean models can make reliable judgments in the lab. Examples of shortcomings include assumptions overriding controls, discounting data that contradicts a preferred interpretation, and miscounting colonies. In one colony counting example, Astra’s counting marks resulted in an overcount of approximately 12% relative to the benchmark reference.
OpenAI states that progress on these tasks could help scientists spend less time troubleshooting and more time deciding which experiments to pursue next. Tracking these capabilities will also help OpenAI understand their implications for biosafety and biosecurity.
Sources
This article was drafted with AI assistance and checked against the sources above. Company claims are reported as claims. Cover image is AI-generated.
The day's checked news, by email
One short email a day with the top stories. Free, and you can unsubscribe in one click.
Or follow for the latest news:
No comments yet. Start the discussion.
Airbnb plans to launch an AI travel agent in the first half of 2027 to help users plan and book trips, the company said in its Q3 2026 earnings shareholder letter.
Censys identified more than 800 publicly accessible Artex deployments, with a separate count of 1,465 internet-facing hosts, some hosting attack tools.
