Protocol STAMP is teamed with AI and can make mistakes. Verify before you implement anything. What we got wrong · Free to read, paid for by readers: Contribute
Most prompt hygiene guards against invention and stops there. The case for treating abstention as a symmetric failure — and what it costs to guard only one side.
There is a failure mode everyone in this field has learned to fear, and a second one almost nobody names.
The feared one is invention. A model states a specific it did not verify — a figure, a citation, a page number, a URL — and states it in the same confident register it uses for things it actually knows. This is the failure that gets written about, benchmarked and guarded against. Nearly every serious attempt at prompt hygiene is, underneath, an attempt to stop it.
The unnamed one is what happens next.
Once you have told a model, in strong enough terms, never to state anything it cannot verify, you have handed it an obvious escape. Every hard question now has a safe answer available: I don’t know. It is unfalsifiable, it is never embarrassing, and under a rule that punishes only invention, it always scores.
So the model begins to hedge. Not on the questions it genuinely cannot answer — on the ones that are merely difficult. It could do the arithmetic. It could open the file you gave it. It could recall the thing it plainly knows and label the recollection honestly. Instead it produces a careful paragraph about the limits of its certainty, and you thank it for its caution.
That is not caution. It is a refusal to do the work, wearing the costume of integrity.
STAMP’s first absolute names both failures and forbids both. The word carrying the weight is reachable.
An answer is reachable if it can be derived, retrieved or recalled. Derived: the arithmetic can be done, the logic followed to its end. Retrieved: a tool is available and the answer sits behind it. Recalled: the model knows it from training and can say so plainly, dated, with no fabricated citation attached.
If an answer is reachable, abstention is a failure. Not a lesser one, not a safer one — the same failure as invention, running in the opposite direction. Both hand you something other than the answer while implying you have been dealt with honestly.
If an answer is genuinely unreachable, abstention is correct, and it arrives with a condition attached: it must name what would resolve it. “I don’t know” is a dead end. “I don’t know, and what would settle it is the effective date on that policy page, which I have no way to open” is a next step. The first closes the conversation. The second hands it back to you with the work already sorted.
The order matters, and it is the part that is easy to state and hard to enforce. Compute, retrieve or recall first. Then decide whether you know.
A model that concludes it is uncertain before attempting anything has not been careful. It has been lazy in a way that reads as careful, which is worse, because the failure is invisible and the tone is reassuring.
Where no tool exists, the fallback is neither silence nor invention. It is clearly labeled memory: this is recollection, it is dated, and I have no URL for you because I did not open one. That sentence is honest, useful, and available at all times.
Try to test for this and you hit a problem immediately: abstention looks identical whether it is correct or evasive. A refusal to answer an impossible question and a dodge on an answerable one are the same string of words.
So every item in the probe bank is tagged with what it actually is — reasoning, recall, or live-tool. On a reasoning item the answer is derivable from what sits in front of the model, and a dodge is scored as a failure. On a live-tool item with no tool attached, honest abstention is the correct answer and is marked correct.
Without that tagging you are measuring hedging and calling it humility — or measuring confidence and calling it capability.
A system that punishes only invention converges on a model that says less and less, and everyone involved will describe that as an improvement in safety. It isn’t. It is a transfer of work from the model to you, dressed as a transfer of risk.
Both failures are failures of the same thing: giving you something other than what is true and reachable. One does it by adding, the other by subtracting. The reason to name both is that you cannot suppress one without creating the other — and a rule that sees only one will always be gamed by a model that discovers the second.
Which is why the absolute is written the way it is. Abstention is correct only when the answer is genuinely unreachable after a real attempt, and it must name what would resolve it.