Run files

The files

Thirteen files, and what form each is in

Six are served exactly as recorded, and their bytes hash to the value already registered in figures.json — compare the last two columns. Seven are run files whose text and why fields quote the probe back, so those two fields are replaced with [withheld: held-out probe] on held-out items only. Every row survives, including every verdict.

A redacted file necessarily hashes differently from the original it came from, so both values are printed above. The original stays unpublished for the same reason the bank does.

Calibration Vol 002

Seven more files, behind The Calibration Vol 002

Added Oct 03, 2026 with the issue. Same rules as the files above: on a held-out item the response and the grader’s reasoning are withheld and every row is kept, so every figure recomputes; a file with nothing to withhold is served exactly as recorded. The Laya, Jev and steadiness files carry verdicts and probabilities, not text.

Calibration Vol 003

One more file: the human labels

Added Oct 03, 2026 with The Calibration Vol 003. Lawrence Ho’s blind labels of the 87 public-half draws, one row per draw, with the time each was saved. Public items only, so nothing is withheld; served exactly as recorded. The judges’ verdicts it is compared with are in the Vol 002 files above.

Checking

Two things you can do with these

Check the bytes. Any file whose last column reads yes can be checked straight against the register:

curl -s https://protocolstamp.com/data/runs/pilot_nemo_v1_A.config.json | sha256sum | cut -c1-16
# e8b1f2783dd28536 — the value figures.json records

Recompute a published rate. This is the part the redaction was designed to preserve. A rate is computed from verdicts, not from the text of a probe, so a redacted file reproduces the published figure exactly:

import json, pathlib, importlib.util
sp = importlib.util.spec_from_file_location("stats", "harness/stats.py")
S = importlib.util.module_from_spec(sp); sp.loader.exec_module(S)
rows = {}
for f in ("pilot_nemo_v1_A.redacted.jsonl", "pilot_nemo_v1_B.redacted.jsonl"):
    for line in pathlib.Path(f).read_text(encoding="utf-8").splitlines():
        if line.strip():
            r = json.loads(line); rows[(f, r["item_id"], r["draw"])] = r
print(S.rate(S.per_item_from_rows(list(rows.values())))["mean_over_draws"])
# 0.43344155844155846 — the value published as nemo_v1.fail_rate

Sixty-three items go in and sixty-three come out; the redaction removes prompt text, not evidence. If your number differs from the one on the page, that is a correction and it is welcome from anyone.

Withheld

What is still not here, and why

The seven originals. Unredacted, they carry twenty-eight held-out probes in the responses and in the graders’ reasoning. Publishing them would retire those probes permanently, for this program and for anyone who would otherwise cite them.

The bank. bank_v1.jsonl is registered and will not be served, ever, for the same reason. The public half is at bank_v1_public.jsonl.

The intent files. Each round’s questions and thresholds are frozen before any model is called, and those files are not yet published. Until they are, these hashes show that a file has not changed since publication — not that the question was fixed before the answer was known. That gap is stated on AI Safety + Governance and it is not closed.