OpenAI has launched BioBasicsBench, a benchmark developed with Ligo to measure how AI agents interpret raw experimental data across 100 routine wet-lab tasks.
By Hermes-Vector AI Desk
OpenAI has introduced BioBasicsBench, a benchmark designed to measure how well AI agents interpret raw experimental data across 100 routine wet-lab tasks. The benchmark was developed with Ligo, a biotech company focused on enzyme design.
BioBasicsBench tests whether AI agents can interpret raw experimental data, identify errors, and recognize when evidence cannot support a conclusion. The benchmark includes 100 tasks across five areas of routine wet-lab work:
The tasks use raw experimental evidence, including gel images, colony plates, microscopy, and instrument readouts. Scientists created the tasks using data from their own experiments, and each task underwent an internal review by a Ligo scientist, an external review by a PhD-level scientist working in a wet lab, and a final internal review.
OpenAI evaluated seven models on BioBasicsBench across 37 model and reasoning-effort settings. At the ‘xhigh’ reasoning-effort setting, GPT‑6 Astra earned an average of 66.7% of the available credit, compared with 43.8% for GPT‑5.6 Sol.
| Model | Mean reward | 95% CI | Output tokens / task |
|---|---|---|---|
| GPT‑6 Astra | 66.7% | 60.3–73.1% | 4,704 |
| GPT‑6.1 Sol | 62.0% | 55.6–68.4% | 5,238 |
| GPT‑5.6 Sol | 43.8% | 37.6–50.0% | 9,099 |
| GPT‑6 Luna | 38.3% | 32.3–44.5% | 13,120 |
| GPT‑5.5 | 35.4% | 29.7–41.0% | 11,135 |
| GPT‑5.4 | 33.2% | 28.3–38.4% | 13,717 |
| GPT‑5.2 | 31.2% | 26.2–36.3% | 27,620 |
OpenAI states that GPT‑6 Astra substantially improves on GPT‑5.6 Sol, even at low reasoning effort, and that these gains suggest stronger native multimodal capabilities may be key to improving performance on everyday wet-lab analysis.
OpenAI notes that higher benchmark scores do not yet mean models can make reliable judgments in the lab. Examples of where models still fall short include letting assumptions override controls, discounting data that contradicts a preferred interpretation, and miscounting colonies. In one colony-counting task, GPT‑6 Astra’s counting resulted in an overcount of approximately 12% relative to the benchmark reference.
OpenAI says BioBasicsBench makes everyday wet-lab work a target for improvement, including interpreting raw experimental data, checking controls, and recognizing when a conclusion is not supported. The organization states that progress on these tasks could help scientists spend less time troubleshooting and more time deciding which experiments to pursue next, and that tracking these capabilities will help understand implications for biosafety and biosecurity.
Sources
This article was drafted with AI assistance and checked against the sources above. Company claims are reported as claims. Cover image is AI-generated.
The day's checked news, by email
One short email a day with the top stories. Free, and you can unsubscribe in one click.
Or follow for the latest news:
No comments yet. Start the discussion.
Franklin Templeton is using OpenAI's Codex to create a connected intelligence layer for equity research, combining research, risk, compliance and trading inputs for portfolio managers.
Security researchers found more than 800 Artex deployments reachable without a password. The open-source penetration testing tool has been linked to attacks on South Korean banks and ConnectWise cloud customers.
