Confidence without false precision
A model that says it is 87% confident — a number it composed in words, not a measurement it read off an internal signal — is usually not calibrated to anything, and rendering that number as a percentage bar launders a guess into a measurement.
HouseWarm renders a per-field confidence number next to an extracted value, and a reviewer uses it to decide where to look first. That is the shipped baseline, it is real human-in-the-loop review, and this lesson is going to ask the question it never had to answer: what is that number a number of?
For an OCR engine the answer may be defensible. Character recognisers emit scores derived from the recognition process itself, and a vendor can tell you how theirs is computed. For a language model asked to rate its own output, the answer is different in kind, and the difference is the whole lesson.
Read the line at the top of this page again with weight on one word: a model that says it is 87% confident. That verb is doing all the work, and the first job here is to keep it doing it.
Two different things are called confidence
Almost every confused argument about model confidence comes from collapsing two mechanisms that share a word. Keep them apart.
Verbalised confidence is the model stating a number. You ask how sure it is, and it composes “87%” the same way it composes any other token sequence. Nothing measured anything. The number is output.
White-box signals are derived from the model’s internals rather than its prose — token probabilities and the quantities computed from them. Something was measured. Whether the measurement predicts correctness is a separate question with its own answer.
Xiong and colleagues tested both at ICLR 2024, across five language models and five reasoning datasets. On the first, their finding is direct: “LLMs, when verbalizing their confidence, tend to be overconfident, potentially imitating human patterns of expressing confidence.”
That is the sentence behind refusing to render a stated percentage as a measurement. The model is producing the register of confidence, learned from text written by people expressing confidence, and register is not calibration.
The part that stops you overcorrecting
The same paper reports that white-box methods still beat verbalised confidence, and quantifies the difference: “the gap is narrow, e.g., 0.522 to 0.605 in AUROC.”
Both halves of that matter. Logprob-derived signals are not the same thing as a model announcing a number, and dismissing them in the same breath would be overclaiming in the opposite direction. But the scale sets the ceiling: AUROC of 0.5 is a coin flip, so a discriminator sitting in that range is weak, not strong. It is the difference between a signal worth ranking a review queue by and a signal worth printing as a probability. It is the first and not the second.
What calibration actually means, and where the definition comes from
The word gets used loosely, so pin it down. Guo, Pleiss, Sun and Weinberger define calibration as “predicting probability estimates representative of the true correctness likelihood”, and report that “modern neural networks, unlike those from a decade ago, are poorly calibrated.”
Scope that carefully, because it is routinely miscited. That paper is ICML 2017 and it studies image and document classification networks. It is not evidence about any language model, and citing it as though it were is the exact move this course spends its time arguing against. What it gives you is the definition, plus the general and important fact that a stated confidence and an actual correctness rate are separable quantities: a system can say 90% and be right 60% of the time, and nothing internal to the system notices.
The definition also tells you what a calibrated number would require: take everything the system rated 0.9, check how often those were right, and see whether the answer is near 90%. That test needs labelled outcomes at volume. If nobody has run it on your pipeline, your component does not have a calibrated number, whatever the number says about itself.
The prior question: should this be on screen at all
Google PAIR’s People + AI Guidebook puts a section heading before any question of visualisation: “Determine if you should show confidence”, with the guidance that if it does not make an impact on user decision making, consider not showing it. The same chapter carries the warning that a misleadingly high confidence may cause users to blindly accept a result.
Google sells AI products, so name the interest: this is published design research from a company with a stake in people trusting AI interfaces appropriately. The guidance is nonetheless the right order of operations, and it is stricter than most teams apply.
The test to actually run is behavioural. Name the decision the reviewer makes, then ask which values of the number change it. If the reviewer reads every step regardless, the number is decoration. If it changes their reading order, you need something ordinal, not something proportional. If it gates an action — below this, a human must approve — then the threshold is the product and the number is an implementation detail that need not be displayed at all.
That last case is what HouseWarm is. The gate is the feature. The per-field number is the mechanism behind the gate, and showing it invites the reviewer to do arithmetic the number does not support.
Confidence does not fix the decision on its own
One more result to hold before designing anything. Zhang, Liao and Bellamy, at FAccT 2020, found that a confidence score “can help calibrate people’s trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making”. Worth stating alongside the result: in their study the humans and the AI had comparable solo performance, which shapes how far it travels.
Set that next to Bansal and colleagues from the highlighting lesson, where explanations increased acceptance regardless of correctness, and the pattern is consistent. Transparency features move trust reliably. They move accuracy much less reliably. A confidence display that shifts a reviewer from reading everything to reading the low-confidence items has changed their behaviour, and whether it improved their decisions is a separate question nobody has answered for your product.
What to render instead
PAIR names four ways to show confidence: categorical, N-best alternatives, numeric, and data visualizations. Which of those is honest depends entirely on the mechanism behind the number, and the mapping below is this course’s own, not PAIR’s.
| What you have | What it honestly supports |
|---|---|
| A model-stated percentage, nothing else | Nothing numeric. Either omit it, or show it as the model’s own words in the register of a quote rather than a metric. |
| A logprob-derived score, uncalibrated | Ordering and coarse bands. Ranking a queue is fine; a bar length is not. |
| A score with bands validated against labelled outcomes | Categorical bands whose labels state the observed rate, and the date it was measured. |
| Several candidate answers with scores | N-best alternatives. Showing the runner-up communicates uncertainty better than any number about the winner. |
Two rules follow from that table, and they are the ones to carry into the component.
The provenance travels with the number. Every confidence value your component renders must carry which mechanism produced it, so the view can pick a treatment and a reviewer can read the label. A schema that types confidence as number has already lost the argument, because the view now has no way to distinguish a stated guess from a measured score and will draw both the same.
Unknown is a value, not a zero. A step with no confidence signal renders as having none. It does not render as low confidence, and it does not inherit a default. This is the same rule as an unset span status in the first module: no claim made is not a claim of failure.
Where people get burned
The percentage-to-bar conversion usually enters a codebase as a one-line utility that turns a value into a width, written by someone who never saw where the value came from. It is nearly impossible to remove later, because by then a stakeholder has seen the bar and reads its absence as the feature being removed.
The cheapest defence is at the type. If confidence is a tagged value carrying its mechanism, the width function cannot be written without handling the stated-percentage case, and whoever writes it has to decide on purpose. If confidence is a bare float, the bar is inevitable.
Retrieval check
A reviewer asks why the panel shows “model-stated: high” instead of the 87% the API returned. What is the answer, in two sentences?
Check your answer
Because 87% is a number the model wrote, not a measurement it read off anything, and no one has checked whether the things it rates at 87 turn out right 87% of the time. Showing the band keeps the ordering the number carries and drops the precision it does not.
The follow-up is worth rehearsing, because it always comes: is the number therefore useless? No, and saying so would be its own kind of overclaim. Xiong and colleagues found white-box signals do beat verbalised confidence, and even verbalised numbers may order items usefully. What none of it supports is proportionality: that 87 is meaningfully more certain than 61 by any particular amount, which is precisely the claim a bar makes and a band does not.
Check your recall
Answer from memory — no scrolling back.
Hands on
Write the confidence rendering decision down before you draw it
Done when: ARTIFACT.md’s “Confidence rendering decision” field names every confidence signal your run emits, the mechanism behind each, what the component renders for it, and one thing it refuses to render, with the reason.
- Inventory the signals. Go through your captured payload and list every value that could be read as confidence, including scores hidden inside tool results and any number a model volunteered in its own text.
- Name the mechanism for each: verbalised, logprob-derived, a third-party service score, or unknown. Unknown is a common and honest answer, and it is the one that changes the design most.
- For any score you did not produce, find the vendor’s own documentation of how it is computed. If they do not document it, write that down. An undocumented vendor score is closer to a verbalised number than to a measurement, whatever the marketing says.
- Write the type. Confidence in your schema carries its mechanism, so the view can branch on it. If the field is currently a bare number, this step is the deliverable.
- State the decision the reviewer makes and which values change it. If no value changes it, the honest outcome of this exercise is to render no confidence at all, and that outcome counts as finishing.
- Write the refusal explicitly: one thing the component will not render, and why. Put it in
ARTIFACT.mdand bring it into the chat. It is the sentence in the whole artifact most likely to come up in an interview, and it is the one that shows judgment rather than craft.
What this does not cover
This lesson refused a number and did not touch the numbers that are genuinely measured. Token counts are emitted per model call, latencies are computed from timestamps, and a dollar figure is those numbers multiplied by a price that lives on a page a vendor can change without telling you. Which of those a component may state, and how it should behave when the price is stale or unknown, is the per-step cost and latency lesson, which opens the next module.
It also stayed out of how a confidence signal gets better. Evaluation harnesses, scoring pipelines and judge models are how a band earns a validated label, and all three are out of scope for this course by design. The course renders what an agent did. It does not grade it.
Read this next — primary source
Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMsMiao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, Bryan Hooi, ICLR 2024; arXiv:2306.13063, June 2023 — free
The paper this lesson rests on, and the reason to read it in full is that it is more careful than the argument people build out of it. It separates black-box elicitation — asking a model how sure it is — from white-box methods that read internal signals, evaluates several prompting strategies and aggregation schemes against each other, and reports where each one lands rather than declaring the whole idea worthless. Read the AUROC tables specifically. They are what stops you from swinging from “render the percentage” to “confidence is meaningless”, which is a different wrong answer, and they set the ceiling on what any confidence UI can honestly promise.
Stuck, curious, or think this lesson is wrong? Ask your teaching agent. The lessons are the scaffold; the conversation is where the learning gets unstuck.