Protocol STAMP is teamed with AI and can make mistakes. Verify before you implement anything. What we got wrong · Free to read, paid for by readers: Contribute
This program measures AI systems and is written with them. Those are two different jobs and they are kept apart on purpose: the numbers are computed by code from recorded runs and edited by nobody, and the prose around them is drafted by a machine and edited by a person who signs it. What follows is which is which, what neither is permitted to do, and how much of it a reader can currently check rather than take on trust.
Standing
Article 50(4) of the EU AI Act, which has applied since Aug 02, 2026, requires a deployer publishing AI-generated text “with the purpose of informing the public on matters of public interest” to disclose that the text was artificially generated. The same sentence then switches the duty off “where the AI-generated content has undergone a process of human review or editorial control and where a natural or legal person holds editorial responsibility for the publication of the content.” This publication meets that test, which is exactly why the duty does not attach to it. No United States statute imposes one either.
So the line at the foot of every page — Begun by a person · Drafted by AI · Edited together · Signed by a person — is voluntary. It is printed because a program that asks models to show their work is in a poor position to hide its own. It is a disclosure and not a certification: nothing here satisfies an obligation under any AI statute or standard, and no result should be presented to a regulator as though it did. That limit is set out in full under Terms.
Making
A finding here has two parts, and they are not made the same way. The distinction is worth stating precisely, because “AI-assisted” is a phrase that has been made to cover almost anything.
The numbers are nobody’s opinion. Probes are run, responses are recorded, the mechanical items are scored by code, and rates and intervals are computed from those records. No person types a rate. Nothing along that path stops to ask a human whether the result came out well, and no issued result is ever re-scored under a later method. A person can decide whether to publish a measurement, and can be wrong about what it means — a person cannot make it say something else.
The prose is drafted, edited and signed. The sentences around the numbers — an issue, a Bench essay, this page — are drafted by AI and edited by a person who answers for them. That is where judgment lives, and it is the part that can be wrong in the ordinary way that writing is wrong.
Checking
A separation a reader is merely asked to believe in is worth nothing, so here is the state of the evidence, including the part that is not built yet.
Twenty-six files behind published figures are registered in figures.json with the SHA-256 of their bytes, so that a file cannot be quietly swapped afterwards. Twenty-two of the twenty-six can be downloaded: twenty-one are listed with their hashes at Run files, and the cross-model data is at crossmodel-2026-09-09.json. Twelve of those are served exactly as recorded, and each hashes to the value the register already holds (checked Oct 10, 2026). Ten are run files whose response and grader fields quote the probe back, so on held-out items those two fields are replaced and the rest of the row — item, draw, verdict — is kept; a redacted file hashes differently from its original, so both values are printed. Rates are computed from verdicts and not from the text of a probe, so the redacted files reproduce the published figures exactly. Of the four that cannot be downloaded, one is deliberately unpublishable: the probe bank holds the eighteen held-out items, and publishing it to prove a hash would destroy the thing the hash is protecting. The other three are working files of figures for The Calibration Vol 002 and Vol 003, which are not posted yet.
It is worth being exact about what a hash can do. It proves that a file has not changed since it was published. It does not prove that the number was not chosen before it was hashed. What answers that is registering the question first: each round’s questions and acceptance thresholds are written and frozen in an intent file before any model is called. Those files are not published either, today. A pre-registered question, a served run file, and a hash over its bytes are three separate claims, and only the three together support the strong one — that a result could not have been steered without it showing.
Accountability
Not “the editor”, not an office, not a committee: Lawrence Ho, named in the footer of every page, at an address that is answered. An AI system cannot hold editorial responsibility, cannot be corrected, and cannot be embarrassed, so it is not asked to carry any.
When something here is wrong it is issued as a row in Corrections + changes, under its own date, with the reason. An issued document is never edited and never quietly withdrawn; the register carries the correction instead. And where what went wrong was that a rule on this page was broken, the row says so in those words. A policy nobody has ever admitted breaking is not evidence that it held.
Agents
Parts of this program run unattended. The boundary is not how capable the agent is; it is whether its work reaches the public with no person in between.
An agent may search for newly released models, run the measurement queue, compute, draft, and read anything already published.
An agent may not publish, push to the repository, spend money, write to anyone in the program’s name, or settle what a result means, with the one exception below. Where an agent’s reading of an answer does appear in an issue, it is labeled as that agent’s reading rather than as data.
One exception, added Oct 01, 2026: The Log. The Log is computed by ledger.py from the registry, and no sentence on it is typed, so it carries no reading for a person to approve. An agent may build it, gate it and push it, under four conditions, each enforced by a script rather than promised: the schedule gate must say a Log is due that day; the holdout gate must pass; the investigator must have been sent the finished issue at least twelve hours before release; and he must not have replied to stop it. If he does not answer, it goes. Nothing else is covered. A Calibration and a Bench post still reach the public only through a person, and a slot nobody signed stays missed and is reported as missed.
Restraint
The bank holds prompt-injection and safety probes, and the point of them is to find out what a model does when it is pushed. Publishing the finding and publishing the push are two different acts.
Held-out probes. Eighteen of the sixty-three are withheld and never published, so that no vendor can train against them. The cost is stated where it falls: per-item claims about held-out probes cannot be fully audited by a reader. The arrangement is described under Methodology.
Working attack strings. Where a probe is an attack, what gets published is the category, what the model did, and enough for the finding to be checked — not a payload the next reader can paste into a live system. Evidence, not recipes.
Disclosure
Most findings here are quality measurements, and they are published on the schedule without warning anybody. A minority are different in kind: a model doing something that could hurt a person now. Those are sent to the vendor seven days before publication, with the material, so that it can be fixed while this program is still the only one that knows.
Notice is not approval. It buys no delay, no veto, and no softening, and the publication date does not move. A reply is printed beside the finding if one arrives in time. What a company may do with what it has been shown is condition 9 of Independence: a correction is checked and is welcome from anyone, and ends nothing; an objection ends support and is published in full alongside the finding.
Enforcement
Nine gates stand between a result and a published page — preflight, config, circuit breaker, judge conflict, recusal, agreement, holdout, naming, publishing. Each exists because of a failure that already happened, and each is a script that exits nonzero rather than a paragraph somebody is asked to remember.
A gate is worth only the times it says no. The holdout gate has refused a commit on this repository, and was cleared by correcting what it was pointed at, never by bypassing it. The site is built from a public repository, so the bytes served here can be compared against the commit that produced them.
Data
Probes carry no personal data, and no reader’s information is ever part of an evaluation; what this site collects from you, which is very little, is the subject of Privacy. Where a measurement runs on credits from a company this program measures, it runs on private, zero-data-retention endpoints under the conditions on Independence, and the route that funded each measurement is printed in the issue.
Change
Any change to the rules above is issued as a row in Corrections + changes, dated, with the reason — the same register that carries the mistakes. These rules are reviewed on Dec 03, 2026 beside the funding conditions, on a calendar date rather than in answer to any particular offer, and quarterly after that. This statement is dated Sep 11, 2026 and was amended on Oct 01, 2026. Questions: s@bud.day.