What happened
- On September 22, OpenAI published a document with its priorities and principles for third-party assessments of frontier models, according to the text hosted on its site.
- It commits to “deep levels of access throughout training, evaluation and deployment” and organizes the work on four fronts: review of safety cases, evaluation of critical safeguards with adversarial testing, capability evaluations for chemical, biological, cybersecurity and self-improvement risks, and investigation of misalignment incidents.
- The principles include a scope agreed in advance, proportionate access within legal and security limits, transparent methodology, evaluators with declared conflicts of interest, and findings delivered with time for remediation before they’re published.
- The document names no evaluating organization. It says it’s in conversations with multiple third parties from the private and nonprofit sectors.
Why it matters
- For any team about to sign with a model provider that needs to justify the decision to a board or to a regulated counterparty, this document is the closest thing to an audit policy that exists today. It’s also worth reading what it doesn’t say.
- Three of the principles (agreed scope, proportionate access, remediation time before publication) leave it to the audited party to decide what gets reviewed, how far anyone looks and when it’s disclosed. It’s a fundamental difference from financial audit regimes, where the scope is set by the rules.
- A consequence the text doesn’t spell out: as long as the evaluators remain publicly unnamed, no organization in the region can verify whether the auditor vouching for its provider has proven experience in the risk that matters to it.
The number
4 is the number of evaluation fronts OpenAI says it’s opening to third parties, without a single evaluating organization identified.
Context
The announcement lands as California sets up a mandatory registry of AI auditors and after Dario Amodei described the pace at which auditors manage to keep up with the labs. The open question remains the same: who accredits the accreditor.
What’s next
- No timelines announced. The document sets no date for naming evaluators or for publishing the first review under these principles.
- The independent evaluation obligations in the European AI Act and the Californian registry are advancing in parallel, on their own timetables.
Bottom line
Publishing an audit’s principles before naming the auditors reverses the usual order. First you learn how things will be looked at; later, whether anyone looked.
Sources
Edited by Rodrigo Cornejo. How we select, verify and correct each note.
