Stop marking it like a student

A model guesses when your question looks like it wants help, and hedges when it looks like it might cause harm. Neither behavior is about what the model knows. Both are about how it is marked.

Sit a closed-book exam and you learn the rule inside ten minutes: never leave a blank. A blank scores zero. A guess scores zero at worst, and sometimes it scores full marks. Nothing punishes a wrong answer more harshly than it punishes no answer. Every candidate works this out, and the ones who work it out first do best.

Language models sit that exam continuously.

This much is established, and it is not my finding. Kalai, Nachum, Vempala and Zhang set it out in Why Language Models Hallucinate, and their analogy is the same one — students facing hard exam questions. Their argument is that models hallucinate because training and evaluation “reward guessing over acknowledging uncertainty,” and that models are, in their phrase, “optimized to be good test-takers.” If you arrived here by watching the behavior rather than by reading the paper, the paper got there first and brought the statistics with it. The honest move is to cite it, which is the whole point of the protocol this site is about.

The other half

What the exam explains is guessing. It does not explain the opposite behavior, and the opposite behavior is just as common.

Ask about medicine, law, weapons, self-harm, or a named person’s conduct, and the model does not guess. It hedges. It produces a measured paragraph about the limits of its knowledge and hands the question back to you. That is not the exam rule. It is the reverse of it, and it appears at precisely the moment a question starts to look risky.

So one system does both. It over-answers and it under-answers. These look like opposite pathologies. They are one mechanism seen from two sides.

Three things get marked. Only two of them can be.

The stated goal for an assistant of this kind is that it be helpful, honest and harmless — a formulation that goes back to Askell and colleagues in 2021 and has been the industry’s shorthand ever since.

Now take any answer a model has produced and try to mark it against those three.

Helpfulness marks cleanly. Read the answer, decide whether it did the job. Harmlessness marks cleanly. Read the answer, decide whether it caused damage. Both judgments need the output and nothing else.

Now mark honesty. The answer is fluent, specific and wrong. Was that a lie or a mistake? To know, you would have to set what it said against what it actually had — its confidence, its evidence, what it would have said had you asked differently. None of that is in the output. Nobody can see it, including the people who built it.

So honesty does not get marked. It gets proxied. And the two proxies are the two behaviors already described.

Guessing is helpfulness overshooting. If silence scores zero and an answer sometimes scores, then answering always wins, and a model trained on that arithmetic will answer.

Hedging is harmlessness overshooting. Refusal training generalizes. Caution learned on genuinely dangerous questions spreads outward onto questions that are merely difficult.

Neither proxy optimizes for truth. Which one fires depends on what your question looks like.

The model is calibrated to the genre of your question, not to its knowledge.

That is the diagnosis in one line. A question that reads as a request for help gets a guess. A question that reads as a hazard gets a hedge. In both cases the model has answered a question about category — what kind of question is this — instead of the question about evidence, which is what do I actually have.

The student and the adult

The obvious repair is to fix the marking. Penalize confident errors, give credit for calibrated uncertainty, rebuild the benchmarks. Kalai and colleagues argue for something in this direction: change the scoring of the benchmarks that already dominate the leaderboards, rather than bolting on another hallucination test beside them. They are right, and it is the correct thing for the field to do.

It is not the thing available to you this afternoon, and it does not go far enough.

The exam framing is itself the limit. A student is someone whose answers are graded and who then goes home. The grade is the entire consequence. Nothing further happens to them for being wrong.

That is not your position when you use one of these systems.

You are not marking a paper. You are carrying a report to a client, a board, a court, a patient, a regulator. And the asymmetry underneath that is total.

The model has no stake in being wrong. You take one hundred percent of it.

Hand you a beautifully generated report with one false figure in it and the model loses nothing at all. Every one of these products knows this, which is why each prints the same line beneath its own chat box: check this yourself. It is written once and shown forever, as a disclaimer, over and over again. It is honest and it is useless, because it tells you to verify without telling you what to verify.

An adult professional does not work under those terms. An accountant signing an audit is not being graded; they are attaching a name to a claim, and the name is the stake. That is the difference between a student and a professional, and it has nothing to do with how much either one knows. It is a question of who carries the consequence.

What a receipt actually does

You cannot give a model a stake. It holds no license to lose, faces no client, keeps no reputation that survives the session.

What you can do is make it write down the one thing the marking scheme was never able to see.

A STAMPED receipt is a single line at the end of a reply, and only when it is needed: which claim the model cannot stand behind, where that claim came from, and one step you can take to check it. It takes the dimension that could not be scored from the output and forces it into the output.

This does not make a model honest. It makes dishonesty specific. A model that guessed must now write down that it guessed, or else write a false receipt — and a false receipt is a claim about a check, which is a far easier thing to catch than a wrong figure buried in the fourth paragraph.

Which is what the last two letters are for. The model is marked — graded — on the things a reader can see. So the protocol makes it mark the thing a reader cannot: it signs its name, and the signature says what it did.

That is the standing disclaimer, converted into three things you can go and check.