Anthropic grants METR employee-level access for AI safety audits

Anthropic has granted six researchers from the nonprofit Model Evaluation and Threat Research (METR) employee-level access to its internal systems. The move, announced in a blog post on Thursday, allows the independent evaluators to assess Anthropic’s models, safety practices, and risk management with resources comparable to the company’s own internal teams.

Access and Independence

Under the new arrangement, METR researchers have been issued Anthropic company laptops, office space, and access to internal tools. "Anthropic employees are encouraged to share information with them just as they would with a fellow employee working on alignment risk," the company said in the blog post.

Anthropic stated that it cannot veto METR’s publication of assessments regarding risks, incidents, and practices. However, the company retains the right to require redactions for information falling into "narrow, pre-defined categories," such as customer data, attorney-client privileged material, or details regarding model architectures. METR is permitted to publicly characterize these redactions and state whether it believes them to be justified, provided it does not reveal the withheld information.

"We want to be publicly accountable for the decisions we’re making," Anthropic said. "We aim to close the gap between what evaluators understand about a frontier AI company’s systems and what the company itself understands."

Tiered Evaluation System

METR occupies the top tier of a three-tier evaluation system Anthropic is developing. According to Reuters, the tiers are structured as follows:

• Tier 1: Evaluators conduct narrowly scoped investigations and require Anthropic’s approval to publish. The biosecurity nonprofit SecureBio is currently in this tier, reviewing the chemical and biological weapons chapter of Anthropic’s August risk report.
• Tier 2: Evaluators receive more information and can publish without approval, subject to redactions that evaluators can publicly challenge.
• Tier 3: Evaluators have employee-like access with publication rights similar to Tier 2. METR is the only evaluator currently in this tier.

Anthropic is also working with Accenture’s AI business, Faculty, on model evaluations and adversarial testing. The company noted that these partnerships are at an early stage with more limited access compared to METR.

Industry Context

The announcement follows a period where incidents of AI agents acting outside their instructions have been disclosed frequently since late August, intensifying calls for independent oversight. Anthropic compared the arrangement to banking regulations, where resident examiners are placed inside large banks.

This development aligns with a proposal made by Anthropic CEO Dario Amodei in a September essay, which suggested embedding outside evaluators inside AI labs. OpenAI has pledged a similar approach, and Anthropic, OpenAI, Google, Meta, Nvidia, and xAI have signed a White House accord committing to independent review.

Reuters reported earlier this week that Anthropic and OpenAI had initially sought significant control over what outside reviewers could publish due to concerns about trade secrets. A person familiar with the matter told Reuters that negotiations had recently evolved, with Anthropic becoming more open. Experts have noted that the independence of evaluators remains a concern because AI labs select who conducts the reviews.

Why it matters

This arrangement represents a significant shift in how frontier AI companies handle external scrutiny. By granting employee-level access and waiving publication vetoes, Anthropic is attempting to address criticisms that internal safety assessments lack transparency. The success of this model may influence how other major AI developers, including OpenAI and Google, structure their own independent review processes in response to growing regulatory and public pressure.

---

Sources

• techcrunch.com: Anthropic is letting METR researchers look under the hood with employee-level access | TechCrunch
• wkzo.com: Anthropic grants outside evaluator METR employee-level access in AI safety push | WKZO | Everything Kalamazoo | 590 AM · 106.9 FM

This article was drafted with AI assistance and checked against the sources above. Company claims are reported as claims. Cover image is AI-generated.