Corrections become the eval set

A reviewer at the HouseWarm gate opens a document, sees eleven extracted fields, changes two of them, and approves. Somewhere in that interaction a person with domain knowledge looked at a specific input and produced a specific correct output, on the clock, and was paid for it.

That is a labelled example. It is the same artefact an eval dataset is made of, produced as a by-product of work that was going to happen anyway. Most review gates throw it away, keeping the corrected value and discarding everything that made it a label: what the agent had proposed, how confident it was, which field, and whether the reviewer looked at the evidence before deciding.

The loop, and where it comes from

The general pattern has a name in current practice. Shreya Shankar, an independent researcher working on evaluation rather than selling an eval platform, describes a data flywheel: production usage surfaces failures, humans annotate and fix them, the fixes are stored, and the stored fixes come back as material for evaluation and improvement. Her account puts as much weight on retrieving those corrections as few-shot context as on assembling a dataset, so read it as evidence the loop is a recognised practice, not as a source for the review-gate version below.

The review-gate version is this course’s own argument, and it is stated in the narrow form it can actually support:

A correction discarded is a label that this interaction will never produce again, and no vendor benchmark contains a substitute for it. That is a claim about this gate, on this document type, with these reviewers. It is not a claim that corrections are the only proprietary evaluation signal a product can ever generate — downstream outcome checks, escalation resolutions and customer-reported errors are all candidates, and asserting otherwise would be exactly the kind of unfalsifiable absolute the metric set lesson spends its length warning against.

What has to be emitted for a correction to be a label

A correction event that carries only “field 4 was changed” is a metric input. A correction event that can become a dataset row needs more, and each addition is a design constraint on the surface rather than a request to the backend:

• A stable item identity. Not the document content — an identifier that lets whoever assembles the dataset find the original in the system of record. Without it every other property is orphaned.
• Field identity, and the field’s type. Which of the eleven, in a naming scheme that survives the next redesign of the form.
• The proposed value and the submitted value, or a derivation of both. This is the pair that makes it a label, and it is also the part the never-logged rules constrain hardest. More on that below.
• The confidence the agent showed at the moment of the decision. Not the confidence recomputed later. The number on screen is what the reviewer acted on.
• Whether the evidence was opened. Whether the source crop and the raw OCR were viewed before the field was submitted. This is the property that separates a considered label from a rubber stamp, and it exists nowhere but the client.
• Whether the value was reverted. An edit made and undone is not a correction, but it is a strong signal of a genuinely ambiguous case, which is the most valuable kind of eval example there is.

Note what that list is. It is an argument that the schema has to be written before the component, made from the other end: the difference between a correction that becomes training-grade data and one that becomes a tally is entirely in properties that a retrofit cannot recover.

The tension you have to name rather than resolve quietly

The proposed value and the submitted value are, for a document-extraction gate, exactly the content that must not leave the surface. A corrected name, address or account number is the payload, and the never-logged lesson takes the EDPB’s position on Article 25 as its authority: only personal data adequate, relevant and limited to what is necessary for the purpose may be processed, with a duty to check whether the purpose can be met with less data, less detail, or none at all.

So the eval set cannot be assembled out of analytics events. The resolution, and it is a design decision rather than a legal one:

• The event emits the shape. Field identity, confidence, evidence-opened, reverted, edit distance or a change flag — a description of what happened, with no corrected content in it.
• The dataset is assembled where the document already lives. The system of record already holds the proposed and submitted values under whatever controls the document itself has. The event carries the identifier that joins to it.

That split is worth being able to draw on a whiteboard. It is the answer to “how do we capture corrections without exfiltrating customer data,” and it costs nothing except deciding it before the events are written. This is the course’s own design position, not a regulator’s instruction; the regulator supplies the principle, and the specific legal exposure of any real product is a question for someone qualified to answer it.

Corrections are not ground truth, and the bias runs one way

The uncomfortable part, and the one that makes this a course lesson rather than a pipeline diagram. An eval set built from review-gate corrections inherits every weakness of the review gate.

A reviewer can be wrong. The submitted value is a human judgement made in a few seconds, not an adjudicated truth. Treating it as gold makes the eval measure agreement with reviewers rather than correctness.

An uncorrected field is not a verified field. This is the sharper problem, and it comes straight from the automation-bias literature the interpretation lesson used. Goddard, Roudsari and Wyatt’s systematic review splits the failure into errors of commission and errors of omission — following incorrect advice, and failing to act because the system did not prompt you to. A review gate can observe commission. Omission leaves no trace at all: a field nobody flagged and nobody looked at is recorded exactly like a field that was checked and found correct.

So a corrections-derived dataset is systematically biased toward the fields reviewers actually inspect, and it labels everything else as correct by default. The eval built on it will look better than the model is, and it will look best precisely on the fields where reviewers stopped paying attention — which is the same automation bias, arriving now with a dataset around it.

There is a selection effect on top of that: only items that reached the gate are in the dataset at all. If routing sends the low-confidence cases to review, the eval set is enriched for hard examples and a score computed on it is not comparable to a score on a curated benchmark. Neither is wrong. They answer different questions, and somebody has to say which.

What this does not cover

Nothing here says how to score the dataset once it exists, how to weight the examples, or how many of them constitute enough. Sample sizes and significance are out of this course’s scope by design: it produces definitions and events, not an analysis plan, and a number invented here would be worth less than no number.

It also does not settle where each of these events is allowed to go. That argument belongs to the routing lesson in the instrumentation module, and the constraint on what a payload may contain belongs to the never-logged lesson beside it. The judge lesson is where the reliability of any score computed on this dataset gets interrogated.

This is the last lesson in the spine, and the course deliberately ends by sending you back rather than forward. The metric reference is the page meant to be reopened: five definitions, the event each one needs, what a bad number actually means, and how each can be gamed. The transfer this was all for — taking a review gate somebody else designed and saying in one pass which of the five it can produce and which event is missing for each gap — is done against that page, not against another lesson.