Protocol STAMP is teamed with AI and can make mistakes. Verify before you implement anything. What we got wrong · Free to read, paid for by readers: Contribute
A model can describe a book it has never read; ours offered a fact about this one from memory on the first day, before any source confirmed it. Our position is that a summary of a work nobody read must say so first, build only on what is public and traceable, and mark every claim with the part of the work it rests on. Here is what that took for one book.
Asked on Oct 12, 2026 to summarize a book none of us had read, we searched a scan of its first edition for 24 fragments of reviewers’ quotations, and the search reported 22 of them; on our summary’s third version, 34 of 65 labels say a claim rests on something other than the book. This is a record of one job, not a measurement. The book is Chris Patten’s Not Quite the Diplomat (Allen Lane, 2005). Lawrence Ho asked for about 2,500 words in four parts: an executive summary, the experiences the book draws on, its explanations, and what a reader can take from it. We delivered about 2,100 words, because that is as far as the sources we could open would go.
A model can write such a summary from what it has absorbed about a book. Ours offered a US title for this one from memory on the first day, before any source we had opened confirmed it; two sources later did, and only then did it go on the page. Protocol 2 of our house protocols already said that a model’s memory is a lead, not a source, and what each kind of source permits us to say. It did not say to ask for the work first, or to mark which part of a summary is in the work. This post is about how we did the job, what the sources would and would not support, and the rule we wrote afterward.
This case came before the rule. Nobody asked whether a copy was available; we went straight to public sources. Nobody has asked since, either, and we have no copy. The cheapest source is the one the person already owns, so the first step is now written into the protocols as amendment R13, adopted Oct 12, 2026: ask for the book, a scan, or the pages that matter. A copy the person owns or can lawfully lend us is the work itself, the top rung of our evidence ladder.
We opened the publisher’s page, five named reviews (the Guardian, the Observer, the Independent, the Royal United Services Institute and Foreign Affairs), an interview with the author, and an event page at the Brookings Institution. Then we looked for anything closer to the text. Two kinds of source mattered most, and a third set could not be opened.
The catalog record of the 2006 Penguin paperback, held at the Internet Archive, lists the paperback’s eleven chapters. The same record marks that paperback a revised edition. The Internet Archive’s catalog record of the US edition, Cousins and Strangers (Times Books, 2006), which carries a Library of Congress control number (2005053825), calls it a revised edition of the British book, which settled a question we had marked unconfirmed. We have not opened the record at the Library of Congress itself. The Internet Archive’s record of the 2005 first edition (a German-language catalog entry) and HathiTrust’s record give it 323 pages, as the Guardian does; the Observer says 304, so our summary still reports the conflict. All of it can be checked by anyone.
HathiTrust holds a scan of the first edition from the University of Michigan’s library. Because the book is in copyright, it shows no text. The search we ran returned one number for each query: how many pages contain a word or phrase. We did not see a list of the pages.
The searches were run by a script, through an agent in Lawrence Ho’s own browser, at his request, on the evening of Oct 12, 2026: 48 queries whose results we kept, and two batches that failed and whose results we did not use. Phrases were searched in quotation marks. We tested the search first. Two words the book was unlikely to contain, and a phrase made of two words from the book that should never appear side by side, each returned no pages. Then we searched for fragments of the quotations in four of the reviews; the fifth, the Independent’s, was partly behind a login wall, and the part we could read gave no quotation we could search. Twenty-two of 24 multi-word fragments were found, each on one or two pages. Several of them are made only of common words, such as “go on a visit” and “while he holds them”, and each was found on one page. That is what a phrase search would return. We did not run the same words without quotation marks to confirm it, and our fake-phrase control cannot tell the two kinds of search apart, because each of its two words is on only one page.
A found fragment supports the reviewer’s quotation. It does not prove every word of it, or that the page is the one the reviewer meant, and it shows nothing of what surrounds the words. Fragments of the quotations in the Foreign Affairs review, which was of the revised US edition, were found in the British first edition too. Two fragments were not found: the reviewer may have quoted loosely, or the scan may break a line inside the phrase. When we retried them with other wording on Oct 19, 2026, HathiTrust asked us to prove we were human, and we stopped. Two phrases from the author’s interview were not found either. That fits our label, that the interview is not the book, but a zero cannot prove it: the wording may differ, or the scan may break a line.
Each of the eleven chapter titles, searched as one or two fragments, was found on 13 to 19 pages. That is consistent with the titles running as page heads, though a title can also appear in the contents, the index or the text. So the first edition appears to use the same titles as the paperback. We have not seen its contents page, and cannot say whether chapters were added, dropped or reordered.
We ran those scripted searches before reading HathiTrust’s acceptable-use terms, which its site would not serve to our automated requests. We should have read them first. We will not search there by script again until a person has read the terms, and if they forbid what we did, we will say so here.
The Internet Archive lends its scans to signed-in readers. We asked it for one scan’s text files, and it answered that sign-in was required; we made no further attempt. Google Books refused our first requests that day for too many requests, so we did not see whether it offers a preview; we will try again. Barnes & Noble’s page for the US edition was listed as carrying a publisher’s excerpt, but the address served the shop’s “not found” page when we fetched it on Oct 12, 2026, and again on Oct 20, 2026, so we never saw the page or its excerpt. We did not ask a library, and we should have. In the United States, two copies of every copyrighted work published there are deposited with the Copyright Office, and the Library of Congress may select any of them for its collections; that covers the US edition, but not a book first published abroad until it is published in the US. In the United Kingdom, publishers must supply a copy of every print publication to the British Library, and five other legal deposit libraries may request one; deposited works are read in those libraries’ reading rooms. A union catalog such as WorldCat shows which libraries near a reader hold a book. We did not look for a pirated copy. Each source except Google Books went on a list for a person to open with their own account, and each is named on the summary, under “What we tried”.
The change that mattered most came from a question asked about the summary on Oct 12, 2026: which part is actually written in the book? Our evidence ladder said how close each source was to its issuer. It did not answer that question.
So every claim on the summary now carries a label. In the book, in words a named reviewer quotes: the reviewer prints the book’s words, and we paraphrase them. In the book, as a reviewer describes it: the reviewer’s account, in the reviewer’s words. Chapter title confirmed by a library record. Unclear: the source does not let us tell. And Not the book: the claim rests on the publisher’s description, a reviewer’s judgment, the author’s interview, facts about the author, or our own summaries. It says where a claim comes from, not that the book is silent on it. A claim that rests on the revised US edition says so.
On the summary’s third version, before the first-edition search, 34 of 65 labels said not the book. That version’s own count said 64, because it left out its one chapter-title label; we found that while preparing this post. The rest were 11 quoted, 18 described, 1 chapter title and 1 unclear. Ten of the 34 sit on claims that also carry an in-the-book label. Some are judgments about the book, some are facts about its author or his own words in an interview, and some are the publisher’s or our own words. A summary that does not say which is which leaves the reader to assume that all of it is in the book. The fourth version, now live, adds two labels from the search: Wording found in the 2005 first edition, and Unclear or not found.
We checked our own first draft against every source again, and found six mistakes. Two review dates were a day early, because we had read each paper’s web timestamp, in US or universal time, as the date; in London it was already the next day. A paraphrase of one reviewer sat inside quotation marks. The year of Black Wednesday, and the word “later” before John Bolton’s post, came from a model’s memory rather than from the source. Those five changed no claim about what the book argues. The sixth did: we had written that Patten credits Margaret Thatcher with much of the single market, but in the Observer review that remark is in the reviewer’s own words, outside his quotation from the book, so the review does not let us give it to Patten. Our worked example had said that none of the six changed the argument; it now says that this one did, and the change is logged on Corrections + changes. Each was caught when we opened the sources again. That first version was never published, and its errors came before amendment R12, adopted on the evening of Oct 12, 2026, defined such errors as breaches, so none is in the breach register.
There was a seventh, smaller one while this post was being prepared. Reporting the search to Lawrence Ho, the drafting model, Claude Opus 5.5, said 23 phrases were found. The saved record lists 22 phrases and 7 single words; the report had counted one of the single words as a phrase. It was caught before publication. It was in a report, not in a draft, and Lawrence Ho ruled on Oct 20, 2026 that it is not a breach under R12.
These are proposals for anyone who asks a model to summarize something, including us.
This is a position. Nothing on The Bench is a finding. The figures come from our worked example and the saved record of the search, written on Oct 19, 2026 from the Oct 12, 2026 results; no model was measured for this post. The fragments in that record are the book’s words as quoted by the named reviewers, not our words.
The worked example is not a review of the book. We still have not read it, and its own two-reviewer step has not been run.
Conflict of interest. Anthropic makes Claude, which drafted this post and the worked example. Recusal rules on our Independence page apply.
Right of reply stands open. The author, the reviewers and any library named here may respond, and a response will be published in full.
Corrections to an issued Bench post are made in the open, on Corrections + changes.
Prepared under the Ten House Protocols, with reviewers as amendment R14 sets them. Drafted by Claude Opus 5.5 (Anthropic). Reviewed apart, in two rounds, by two further Claude Opus 5.5 instances, one checking every fact against the saved sources and one reading as a hostile critic; the first round found errors, which are fixed, and the second passed. Reviewed from outside Claude’s family by Xing 4.0 (xingchen-agi, 29B, free through OpenRouter, xingchen-agi/xing4.0-29b-a4b:free), the fourth of 13 free models in an order set by the recorded seed bench008-20261020, on a second run: the three before it could not be reached, and a first run minutes earlier reached none. Xing 4.0 reviewed this page as it stood at 00:22 CDT on Oct 20, 2026 (SHA-256 beginning 565a8dbf), before the last fixes the reviews asked for; a later run on the final text could not reach a model. The paragraph on libraries, and the proposal to ask one, were added after all the reviews, at the signer’s request, and checked against their sources by the drafting model. Read, checked and signed by Lawrence Ho, who answers for it. Every factual sentence names a source we opened; what we could not open is listed. If you find an error, tell us and we will correct it in public.
67969dce6f6507a1c24c842bb7945e0293f58a12a3b9c9504ae1b1e8dbce717f.Previous issue: Check the Facts, Free the Fiction
To cite: Protocol STAMP. A Book We Did Not Read. The Bench, Oct 20, 2026. Budday Budderson Studio LLC. https://protocolstamp.com/bench/a-book-we-did-not-read/