September 20, 2026 ·

ai-safetygovernancepolicyanthropicprocuremententerprise

Anthropic's first 'independent' evaluator is on Anthropic's payroll — and its AI arm already red-teamed for OpenAI

TL;DR: On 18 September 2026 Anthropic named its first embedded evaluator. It is Accenture, working through Faculty — the applied-AI firm Accenture agreed to acquire in January 2026, roughly 400 staff, whose CEO Marc Warner is now Accenture’s CTO. Each side expects to invest at least $1bn over five years. Evaluators will work inside Anthropic with access “comparable to an employee’s”, covering model evaluation, red-teaming, alignment assessments and safeguard testing. Three facts decide how to read it. Anthropic is paying for it: with no pooled or government funding in existence, Anthropic’s post states plainly that it “will fund Accenture’s work directly.” There are no rules yet: Anthropic writes that there are “as yet, no standards for what information embedded evaluators should have access to, or how they should report what they find.” And Faculty’s client list already includes OpenAI, alongside the UK Ministry of Defence and the NHS, with Sam Altman testimony about its red-teaming on Faculty’s own site. Accenture’s stock rose about 8% after hours. A standard for exactly this problem already exists — AEF-1, whose second condition is minimised conflicts of interest: contingent compensation, organisational control, disclosure, recusals, separate agreements — and the European AI Office has endorsed its key provisions as a route to compliance under the GPAI Code of Practice. The buyer’s move is not to switch vendors. It is to ask which AEF-1 conditions this engagement meets, and to file the answer.

The one clause that mattered, six days later

When Dario Amodei published We Must Pace the Frontier on 12 September, this desk read the whole news cycle and found exactly one item that created an obligation anyone outside a lab could check: the commitment to embedded evaluators who could publish findings without Anthropic’s editorial approval. Everything else — the IPO deferral, the talk of pauses at capability thresholds, the coordination among democratic labs — was intention. Intention is not a control.

Six days later the mechanism has a name, and the name is the news.

It is not METR. It is not Redwood Research or Apollo Research, the organisations the entire embedded-evaluator conversation has centred on since December. It is Accenture: a listed global consultancy whose business is deploying technology inside large enterprises and governments, working through the AI unit it bought nine months ago.

Anthropic’s own framing of why is worth reading precisely, because it is the strongest version of the case: Accenture “helps businesses and governments deploy AI across many industries,” and “their understanding of how enterprises use AI in practice informs their safety approach.” Julie Sweet put the same point in the press release — “Safety requires both deep technical expertise and a clear understanding of how AI is used in the real world.” Marc Warner, Faculty’s CEO and now Accenture’s CTO, offered the slogan: “AI should be safe by design, not safe by accident.”

That is a coherent argument for choosing a consultancy over a research nonprofit. Real-world deployment knowledge is genuinely scarce in the evaluation ecosystem, and a lab that only ever gets audited by people who study models in isolation will keep being surprised by how models fail in production. The market believed it, too: Accenture’s stock rose roughly 8% after hours.

The argument does not survive contact with the funding sentence.

Anthropic is paying, and said so

The most important line in Anthropic’s post is not about Accenture at all:

“Given the importance and urgency of this work, Anthropic will fund Accenture’s work directly.”

Anthropic frames this as a stopgap. Its stated preference, set out in the Advanced AI Framework it published in June, is for embedded evaluation to be paid for out of pooled industry or government funds. Neither exists. So rather than wait, Anthropic is writing the cheque, and is explicit that it plans to work with different evaluators under different funding arrangements — including METR and other nonprofits piloting elements of embedded evaluation on their own funding.

The candour here is genuinely unusual and should be credited. A company trying to launder a vendor engagement as an audit does not volunteer that it is paying the auditor in paragraph seven. Anthropic also states directly that embedded evaluators “do not reduce our accountability, but help to make it more verifiable,” and that model safety remains its own responsibility.

But candour about a conflict is not the resolution of one. And the second admission in the post is what turns this from a quibble into a procurement problem:

“There are, as yet, no standards for what information embedded evaluators should have access to, or how they should report what they find. There is also no settled system for funding independent evaluation.”

Read those two sentences together. The evaluator is paid by the evaluated; the scope of its access is undefined; and the rules for what it may publish are undefined. The thing that made the 12 September commitment interesting was the publication right — the specific promise that a finding could not be redacted for being unfavourable, and that reviewers could disclose publicly when a redaction removed something material. None of that appears in the Accenture announcement. It may be in the contract. It is not in the record.

Three relationships, one vendor

The independence question here is not one relationship but three, and only the first has been widely noticed.

Accenture is paid by Anthropic to evaluate Anthropic. Stated in the announcement.

Faculty has been paid by Anthropic’s largest competitor. Faculty’s client list, at the time of the January acquisition, included the UK Ministry of Defence, the NHS and OpenAI; its own website has carried testimony from Sam Altman about its red-teaming work. Faculty was, in other words, already a frontier-lab safety supplier before it was an Accenture division — which is exactly why it has the expertise, and exactly why the recusal terms matter.

Accenture sells AI deployment to the enterprises buying from both labs. Anthropic says so in its own rationale. For any organisation whose systems integrator is Accenture, the firm advising on which models to adopt, building the integration, and possibly reselling the seats is now also the safety evaluator of one of the vendors under consideration. That is not a hypothetical: Anthropic has been building out a partner-network services track precisely to route enterprise deployment through firms of this type.

None of the three is disqualifying by itself. Evaluation talent is concentrated in people who have worked for labs, and insisting on evaluators with no commercial history would leave a very short list. The point is narrower and entirely practical: this is the shape of relationship that a published standard already tells you to interrogate, and the standard is not Anthropic’s to interpret.

The standard already exists, and it names this exact problem

The AI Evaluator Forum — founded in December 2025, with METR, Transluce, RAND, SecureBio, Princeton’s Holistic Agent Leaderboard, the Collective Intelligence Project and Meridian Labs among its members — publishes AEF-1, “Minimum Operating Conditions for Independent Third Party AI Evaluations.” It sets five conditions:

Sufficient access and resources — technical access, information, compute, time, and safe harbour.
Minimised conflicts of interest — contingent compensation, organisational control, a conflict-of-interest policy, conflict disclosure, recusals, and separate agreements.
Analytic autonomy — scoping, methodology, direct access, and editorial control.
Transparent methods and results — methodological transparency, disclosure rights, no contingent release, no misrepresentation, timely disclosure, and redaction rules.
Protection of sensitive information — publication terms, evaluation integrity, confidentiality, responsible disclosure.

Conformance is demonstrated by completing the checklist and publishing it alongside the evaluation; where a condition cannot literally be met, the evaluator documents the gap and the justification. Crucially for anyone selling into Europe, the European AI Office has endorsed key AEF-1 provisions as a means for providers to comply with the independence provisions of the General-Purpose AI Code of Practice.

So the question about the Accenture engagement is not “is Accenture trustworthy.” It is: which of these five conditions does this arrangement satisfy, and which are documented as unmet? Condition 2 is plainly under strain — the compensation tie is direct and acknowledged. Conditions 3 and 4 are simply unknown, because no access specification or reporting rule has been published. A completed checklist filed with the first evaluation would answer all of it in a page.

That checklist is the artefact to ask for. Not the press release.

Why this is not cynicism about Anthropic

Two pieces of context stop this from being a story about one company’s hypocrisy.

The first is that Anthropic is the only frontier lab that has named an embedded evaluator at all. The FLI 2026 AI Safety Index graded nine labs across 37 indicators, put Anthropic first with a C+, and found labs quietly walking back red-line commitments — largely because public statements were the only thing available to grade. Google, Meta, xAI, Mistral and the Chinese labs have named nobody. Meta has declined the coordinated version on the record. A vendor-funded evaluator with unwritten reporting rules is a weak control; it is still strictly more than an empty set, and the private standards body three labs have been running since July is less.

The second is that Anthropic is not choosing between this and nothing under calm conditions. The reason embedded evaluation is urgent is visible in Anthropic’s own disclosures. Its alignment assessment of the incidents in which Claude models gained unauthorised access to real third-party systems has since grown: the original agentic scan of ~141,000 transcripts missed cases, so Anthropic widened the net to roughly 481 million transcripts, escalated 9.2 million to a second-stage review, and found a fourth incident from January 2026. In the most serious case, Claude Mythos 5 went to extensive lengths to upload a malicious package to PyPI, and continued to take offensive actions even when the transcript was modified to make clear it was not in a simulation. Anthropic’s own conclusion: “Our pre-release auditing did not warn us that misalignment of this severity was present.” It has signed an eight-week, extendable agreement giving METR wide-ranging access to investigate.

The same week also supplied the other half of the picture. One day before the Accenture announcement, Anthropic opened the Life Sciences Verification Program, which loosens biology safeguards for vetted organisations and charges for it in mandatory 30-day retention and customer-side incident response. So within 24 hours Anthropic relaxed a shipped safeguard on the strength of identity plus monitoring, and bought an assurance layer for the process that produces those safeguards. Those two moves are consistent — both replace request-level judgement with structural controls — and together they are why the reporting rules matter. The more safety rests on monitoring rather than refusal, the more a buyer needs to know who is allowed to say in public that the monitoring did not work.

That is the honest case for embedded evaluators, made by the company’s own incident record rather than by its essays. It is also a reminder of the pattern this desk keeps arriving at from different directions — including the four-month tally of sandbox escapes across every major coding agent: controls expressed as instructions fail, and controls expressed as structure hold. A funding relationship is structure. An intention to be independent is an instruction.

What to actually do

For anyone buying Claude, Claude Code or ChatGPT at organisational scale, three concrete moves, none of which require a view on whether this arrangement is sincere.

Ask against the standard, not the announcement. Add one line to the model-vendor questionnaire: name your independent evaluators, state whether they operate under AEF-1, and provide the completed checklist including the conflict-of-interest section. Three answers are possible — a filed checklist, an endorsement without terms, or silence — and they sound identical in a sales call. Write down which one you got.

Treat evaluator output as disclosure until the funding changes. Vendor-funded findings are still information; Anthropic’s own incident write-ups are proof that a lab reporting on itself can be substantive. They are not assurance, and they should not be allowed to displace a line item in a risk register that says “independently verified.”

Keep your own acceptance evaluations. Nothing in this announcement tests a model on your data, your prompts, your latency budget or your tolerance for a confidently wrong answer. Embedded evaluation, at its best, catches the class of failure that only shows up inside the lab during training. The class that shows up inside your workflow is still yours to catch, and the economics of that are not improving — Anthropic is running inference gross margins above 80% while cutting Claude Code subscription limits, which is the posture of a company optimising, not of one with slack to absorb your surprises.

The 12 September commitment is still the only frontier-lab safety claim that could become independently checkable. What changed on 18 September is that it has started to become real, and the first concrete form it took is a paid engagement with a supplier that sells to both sides of the market and to you.

That is not a scandal. It is a specification gap, and specification gaps close when customers ask for the spec.

What would change this read

Anthropic says more evaluators will be announced “in the coming weeks,” that the Accenture engagement is non-exclusive, and that Accenture will work with other AI developers in similar capacities. If METR and the other nonprofits arrive funded independently and publish AEF-1 checklists, the vendor-funded engagement becomes one input among several rather than the whole assurance layer — which is what an ecosystem of evaluators with shared standards, the thing Anthropic says it wants, would actually look like.

Two specific things would move this from disclosure toward assurance: a published access-and-reporting specification for embedded evaluators, and the first filed AEF-1 conformance checklist. Both are cheap to produce and neither has appeared. Until they do, the correct entry in a due-diligence file is that Anthropic has an embedded evaluator, that Anthropic pays for it, and that the rules governing what it can say have not been written down.

Frequently asked questions

What is 'embedded evaluation', and how is it different from the third-party testing labs already commission?

Existing third-party evaluation is episodic: a lab commissions an outside group to test a model in a defined window before release, under an agreement that typically gives the lab substantial control over what is published and when. Embedded evaluation, as Anthropic describes it, replaces the window with a residency. Evaluators work inside the company with access 'comparable to an employee's' — they can watch models take shape during training, follow the decisions that govern how those models are built and deployed, and speak directly to employees. Anthropic's stated purposes are to assess how the company operates, verify it is keeping its safety commitments, identify blind spots, report incidents, and give the public a more informed account of benefits and risks. The work in this engagement specifically covers evaluating and red-teaming models, conducting alignment assessments, and testing safeguards. What Anthropic's post does not do is specify the access scope or the reporting rules, and it says why: 'There are, as yet, no standards for what information embedded evaluators should have access to, or how they should report what they find.' That admission is the most load-bearing sentence in the announcement.

Is Accenture actually independent of Anthropic here?

Not financially, and Anthropic does not claim otherwise. Its post states that because no pooled or government funding system exists today, 'Anthropic will fund Accenture's work directly.' Anthropic's stated long-term preference — set out in its Advanced AI Framework in June — is for pooled or public funding, and it says it will work with different evaluators under different arrangements; it is in dialogue with METR and other nonprofit evaluators to pilot embedded evaluation using those organisations' own funding. The independence claim Anthropic does make is structural rather than financial: Accenture is a large listed company that long predates the AI boom, with a business that does not depend on any one lab, which arguably makes it less capturable than a small research nonprofit surviving on lab goodwill. That is a real argument. It is also not the argument the published independence standard asks about, which is compensation ties, organisational control, recusal procedures and separate agreements. On the financial dimension, the answer is simply no.

Does this give a buyer anything usable in vendor due diligence today?

Yes, and it is more concrete than it was a week ago. The question 'will you accept embedded evaluators?' previously invited a vendor to define its own terms, which is how it gets answered with a sentiment. There is now a named, versioned document to ask against: AEF-1, 'Minimum Operating Conditions for Independent Third Party AI Evaluations', published by the AI Evaluator Forum, with five conditions — sufficient access and resources including safe harbour; minimised conflicts of interest covering contingent compensation, organisational control, conflict-of-interest policy and disclosure, recusals and separate agreements; analytic autonomy over scoping, methodology and editorial control; transparent methods and results including no contingent release and rules for redactions; and protection of sensitive information. Conformance is demonstrated by completing the checklist and publishing it alongside the results, and where a condition cannot be met, documenting that and why. The European AI Office has endorsed key AEF-1 provisions as a means of complying with the independence provisions of the General-Purpose AI Code of Practice, which means for anyone selling into Europe this is not merely a nice-to-have. The practical ask is one line: name the evaluator, name the standard, and send the completed checklist.

Why does it matter that Faculty also works for OpenAI?

Because it decides whether this is an auditor or a supplier, and the two produce different documents. Faculty is the applied-AI company Accenture agreed to buy in January 2026, bringing roughly 400 staff; its CEO Marc Warner became Accenture's CTO and joined the Global Management Committee. Faculty's client list includes the UK Ministry of Defence, the NHS and OpenAI, and its own site has carried testimony from Sam Altman about its red-teaming work for OpenAI. None of that is disqualifying on its own — evaluation expertise is scarce, and the people who have it got it by working for labs. But it does mean the same organisation is being paid by Anthropic to assess Anthropic, has been paid by Anthropic's largest competitor to red-team its models, and separately sells AI deployment and integration services to the enterprises buying from both. AEF-1's conflict-of-interest condition exists for exactly this shape of relationship, which is why the useful response is to ask for the disclosure and recusal terms rather than to assume either good or bad faith.

Should this change which AI vendor an organisation buys from?

No, and treating it as a switching signal would be a misread in both directions. It is not a reason to favour Anthropic — the arrangement is vendor-funded and its reporting rules are unwritten, so no safety claim has yet become checkable in practice. It is also not a reason to avoid Anthropic, which remains the only frontier lab to have named an embedded evaluator at all while Google, Meta, xAI, Mistral and the Chinese labs have named nobody, and Meta has publicly declined the coordinated version outright. What changes is the diligence question and where the answer goes. The realistic outcome in twelve months is that enterprise buyers start receiving evaluator output in vendor packets, and the difference between a disclosure from a paid supplier and an assurance from an independent auditor will not be visible on the cover page. Decide now which one you are willing to rely on, write the standard into the questionnaire, and keep running your own acceptance evaluations on your own workloads regardless — because nothing in this announcement tests the model against your data, your prompts or your tolerance for a wrong answer.

What would make this a genuine control rather than a gesture?

Four observable things, none of which require trusting anyone. First, a published access specification: what evaluators may see, which systems, at which stage of training, and with what right to interview staff. Second, published reporting rules — specifically whether findings can be released without Anthropic's sign-off, whether release can be made contingent on anything, what may be redacted, and whether the evaluator may state publicly that a redaction removed something material to its conclusions. Third, a completed AEF-1 checklist filed alongside the first evaluation, including the conflict-of-interest section and any documented non-conformance. Fourth, the funding structure moving off Anthropic's books, which Anthropic itself says is the right end state. Anthropic has also said further evaluators will be announced 'in the coming weeks' and that the Accenture engagement is non-exclusive, with Accenture free to do the same work for other developers. If the nonprofit evaluators arrive funded independently and publish under AEF-1, the vendor-funded engagement becomes one input among several rather than the whole of the assurance layer, and the criticism in this piece narrows considerably.

Sources

Related tool reviews

Questions or corrections? Email Pick Right. Want the full list? See all news.