Loading
Five numbers a review gate can produce, with the events required to compute each one, what a bad reading actually means, and how each can be moved without touching the agent. This is the page to keep open while filling in learning/agent-evaluation/INSTRUMENTATION.md, and the page to re-read before quoting any of these numbers to somebody who will act on them.
One of the five — time-to-trust — is not a standard term. Its entry says so, and so should you, every time.
Descriptive, computable, and the load-bearing metric of the set — every other number here is either derived from it or interpreted against it.
Numerator: reviewed items in which at least one field value the reviewer submitted differs from the value the agent proposed. Denominator: items that reached a terminal reviewer decision. Window: rolling 28 days, bucketed by decision timestamp. Population: reported per reviewer and pooled, never pooled alone. Keep a second, field-level variant — fields changed over fields presented — and never let one be quoted as the other. Edit-and-revert does not count here; the comparison is against the submitted value, and the revert is captured as its own event.
review.item_opened — item id, agent output hash, field count, per-field confidence bandreview.field_changed — field id, changed-from-agent-value flag, confidence band of that fieldreview.field_reverted — field id, time since the original editreview.decision_submitted — outcome (accepted_unchanged | accepted_corrected | rejected | escalated), fields changed, dwell before submitA high correction rate is not automatically bad: it can mean the agent is weak, or that the gate is correctly catching a genuinely hard class of documents, or that a field is systematically mis-specified rather than mis-extracted. A low one is the dangerous direction, because it is the one everybody wants. Falling correction rate is equally consistent with an improving agent and with a disengaging reviewer, and the metric alone cannot separate them. Split by confidence band before believing either story: corrections concentrated in low-confidence fields is a functioning gate; corrections spread evenly across confidence bands means the confidence signal is not carrying information.
Trivially, by widening what counts as a correction (normalisation, whitespace, casing) so the numerator inflates, or by narrowing the denominator to only the documents that reached a decision on the happy path. Less trivially and far more often: by shipping any interface change that makes approving cheaper than scrutinising — a default focus on the approve button, an approve-all affordance, a keyboard shortcut with no confirmation. None of those is dishonest, all of them move the number, and none of them touches the agent.
Descriptive and computable, but only meaningful when derived from the same mutually exclusive partition as correction rate.
Numerator: items approved with zero field values changed. Denominator: items that reached a terminal reviewer decision. Derive it, correction rate, rejection rate and escalation rate from one partition — accepted unchanged / accepted with corrections / rejected / escalated — so the four shares sum to one and definitional drift shows up as a total that fails to close.
review.decision_submitted — the same event as above; acceptance is a projection of its outcome field, not a separate emissionreview.evidence_opened — whether the source crop, raw text or per-field detail was ever opened before the decisionDefined independently rather than as part of a partition, acceptance rate double-counts every corrected-then-approved document and the rates sum past 100%; because each is individually plausible, this survives review. A rising acceptance rate reads as improving output and reads equally well as a reviewer whose bar for “good enough” has drifted, or as an interface that made approving structurally cheaper than checking. Pair it with review.evidence_opened before drawing either conclusion: acceptance rising while evidence-opened falls is the disengagement signature.
By reclassifying corrections as something else — an “enrichment,” a “formatting fix,” a “post-processing step” — so they leave the correction bucket without leaving the workflow. Also by moving the gate later, so the easy documents are auto-approved upstream and only the hard ones reach a human: acceptance rate on the residue is a different population being reported under the same name.
Not a standard term. This course’s own construct — say so every time you use it, in the same sentence.
This course’s definition: the number of completed review sessions after which an individual reviewer’s per-session correction rate first drops below a stated threshold and stays below it for three consecutive sessions. Sessions, not calendar days — calendar time measures rostering. Per reviewer, never pooled. The threshold and the three-session run are choices, not findings; state them wherever the number is quoted.
review.session_started / review.session_ended — reviewer id, item count, elapsed active timereview.decision_submitted — everything above, since the construct is derived entirely from correction rate per sessionThe number is a derived metric resting on a contested one, and it names an internal state it never observes. A short time-to-trust is indistinguishable from a system that invited unearned trust quickly. Buçinca et al. found self-reported trust positively correlated with overreliance on incorrect model predictions — so even the honest, validated versions of this construct can move opposite to decision quality. Treat any favourable time-to-trust curve as a prompt to look at evidence-opened and dwell, not as a result.
By choosing the threshold after seeing the data, which converts an arbitrary parameter into a conclusion. Also by pooling reviewers: one high-volume reviewer who never corrects anything can pull a team construct into the flattering range. The most common form is neither — it is quoting the number without disclosing that the definition is local, so a house construct is heard as an industry benchmark. There are no industry benchmarks for this term.
Computable only from client-side events. On a backend that hears only about approvals, this metric silently reads zero.
Numerator: review sessions in which the reviewer opened an item, produced at least one interaction with it, and reached no terminal decision before the session expired. Denominator: review sessions in which an item was opened. Bucketed by session start. State explicitly whether “mid-run” means mid-review (a request/response gate like HouseWarm’s, where extraction has already finished) or mid-agent-run (a streaming, multi-step surface). The two are not comparable and will share a name across a portfolio.
review.item_opened — as abovereview.interaction — coarse activity heartbeat: scrolled, focused a field, opened evidence (a signal that work is happening, not a keystroke log)review.session_ended — terminal decision reached or not, reason (submitted | timeout | navigated_away | closed)Falling abandonment reads as a gate that got easier to finish, and reads equally as reviewers who stopped hesitating over cases that deserved hesitation. Hesitation-then-completion and no-hesitation-at-all are identical in a completion metric, which is why dwell has to be captured alongside it. A stable zero almost always means the events are missing rather than that nobody ever walks away — check that before celebrating it.
By extending the session timeout, which converts abandonments into long-running sessions. By auto-saving a draft decision, which converts them into completions. By counting only sessions that reached the gate through the primary route. Each is a defensible engineering choice with a metric side effect, which is exactly why the definition has to pin the timeout and the reason codes rather than inheriting them from whatever the session library does.
Entirely conditional on an affordance existing. Without one, the metric is not low — it is undefined and reported as zero.
Numerator: items the reviewer routes to a named exception path — second reviewer, supervisor queue, explicit can’t-decide state. Denominator: items reaching a terminal reviewer decision, with escalation counted as one of the four terminal outcomes. Record the reason alongside the route, or the number tells you volume and nothing else.
review.escalated — target route, reason code, dwell before escalating, whether evidence was opened firstreview.escalation_available — emitted with the item, recording whether the affordance was offered at all for this item and reviewerZero is the reading to distrust first: it usually means no affordance exists, which is a fact about the component tree, not about reviewer confidence. A falling rate is equally consistent with fewer ambiguous cases and with an escalation path that acquired a social cost — escalation is the metric in this set most sensitive to organisational pressure that has nothing to do with the agent. A rising rate is not a failure signal either: it can be the gate working exactly as designed on a harder input distribution.
By removing or burying the affordance, which makes the number improve immediately and permanently. By adding friction to the escalation path — a mandatory free-text justification, a notification to a supervisor — which suppresses it without removing it, and is harder to spot afterwards. Both are the same error the EU AI Act’s Article 14(4) is aimed at: oversight is only real if the person doing it is genuinely enabled to disregard, override or reverse the output.
Every one of these numbers is computed from a human’s behaviour and reported as a fact about a model. The flattering reading is always a claim about the agent; the competing reading is always a claim about the reviewer. Before quoting any of them, say which one you are making and what would have distinguished it from the other.
Catalogues thirty measures of trust in automation. None of them is a “time-to-trust,” which is the basis for treating that term as a local construct.
Self-reported trust was positively correlated with overreliance on incorrect predictions; mental demand was positively correlated with performance on those same cases.
Explanations increased acceptance of the AI’s recommendation regardless of its correctness, with no significant accuracy gain over a bare confidence score.
Systematic review of 74 studies; risk ratio 1.26 (95% CI 1.11–1.44) for following erroneous advice. Source of the commission/omission split.
Survey of 41 human-oversight policy documents; none defines “meaningful” oversight, and the evidence base for effective human oversight is thin.
Names automation bias in the statute and requires the overseer be enabled to disregard, override or reverse the output, and to interrupt the system. A reading copy of Regulation (EU) 2024/1689, not the Official Journal.
Every claim on these pages links to its source. If a source looks wrong or out of date, check the resource list and tell your teaching agent — the course is meant to be corrected.