What the agent changed

The run has finished. Somewhere, something is different: a file, a row, a field on a record, a message that left the building. Everything in this course so far has described what the agent did. This lesson is about what it changed, which is a smaller set and a more consequential one.

It is also the lesson where this course has to correct itself in public, so start there.

The claim this lesson does not make

The tempting version of this argument is that a diff is the one view that lets a reviewer approve or reject without reading the reasoning at all. It is a satisfying line and there is no evidence for it.

A search for empirical work on diff-based review of AI-generated changes — covering diff review, AI-generated changes, human review, approve-or-reject decisions, and reviewing without reading a stated rationale, across general web search and arXiv — turned up nothing establishing that reviewing a diff alone is safe or sufficient. What it did turn up points the other way. Duma and colleagues, studying review activity on AI-generated pull requests in the AIDev dataset against human-authored pull requests in the same repositories, report that “most AI-generated PRs receive no review and, when reviewed, are largely dominated by AI agents rather than humans”, with human-authored changes drawing direct human feedback while AI-authored ones draw review that “more often take[s] the form of automation-mediated interaction.” They caution against reading review metrics as a proxy for human oversight.

That is a finding about pull requests on GitHub, not about your panel. It does not prove a diff view causes shallow review. It does remove the cover story, which was that a good enough diff makes the reasoning optional.

Define the change surface before you render it

The load-bearing rule of this course is that you learn the data shape before designing the view, and it applies here with a twist: the shape you need is not emitted by the runtime at all. Nothing in the span model tells you what a run changed. Spans record operations, not effects. A tool call that wrote four hundred rows and a tool call that wrote none can produce identical spans.

So the change surface is something you enumerate, per product, before you draw anything. For your runs, which of these can an agent touch?

• Files in a working tree
• Rows or documents in a datastore
• Fields on a single record, the HouseWarm case
• Messages sent to a person or a queue
• Calls to a third party that has its own side effects

A diff component built against the first item on that list renders beautifully and hides the last three. This is the same failure the opening lesson of the course described for trace panels, arriving one module later in a new costume: a view designed against an imagined payload, which handles the case its author pictured and drops the rest.

Reversible and irreversible are not the same row

A diff carries an implicit promise. Showing before and after implies that before is somewhere you could go back to. For a file edit that is roughly true. For an email that has been sent, a payment that has cleared, or a webhook a third party has already acted on, it is false, and the interface has made a promise the world will not keep.

This course’s position, again as a position and not a citation: an irreversible effect does not belong in the diff table. It belongs in a separate list, visually distinct, labelled as already done, ordered by when it happened. A row you cannot reject is not a row in a review queue. It is a receipt.

The distinction also decides where your approval gate goes. If the agent’s irreversible effects run before the human sees the diff, the diff is a report and calling it a review gate is a category error. Gate first, or say plainly that you are not gating.

Every row points back at a step

This is where the diff joins the rest of <AgentTrace /> rather than sitting beside it. Each change carries the id of the step that produced it, and the row links to that step in the tree. One click from “this field changed” to “here is the tool call that changed it, its arguments, its result, and the model call that decided to make it.”

That link is the whole reason to have built the tree, and it is the answer to the paper above. The finding was not that humans review AI-generated changes badly because diffs exist. It was that review of agent output looks structurally different from review of human output. A diff whose every row can be traced in one click to the step that caused it is the version of this UI that makes deeper review cheap rather than making shallow review comfortable. That is a design bet, and stating it as a bet is more useful than dressing it as a result.

The join also constrains the schema. If your captured payload has no stable step identifier on the record that describes an effect, this link cannot be built, and that is a finding about the runtime worth writing into the artifact ledger rather than a problem to route around in the view.

Approval has to be divisible

One more position, and it follows from watching what people do with gates rather than from any study: if the only controls are approve all and reject all, reviewers approve. A run of thirty changes containing one wrong change is a run where rejecting everything costs the reviewer the twenty-nine correct ones, and that cost is paid immediately while the risk of the wrong one is hypothetical.

Per-row accept and reject changes the arithmetic. It also produces something more useful than a decision: a record of which rows a human rejected, per step, which is the closest thing to ground truth this whole component can generate about where the agent goes wrong.

What this does not cover

This is the last lesson, so the honest version of this section is a list of what the course leaves you holding rather than a pointer to a sibling.

The diff lesson does not tell you how to compute a diff for a datastore that keeps no history. Capturing a before requires a snapshot taken at the right moment, which is a runtime concern this course deliberately stayed out of — it renders telemetry rather than operating it, and that boundary was set in the mission for a reason.

It also leaves the harder version of the review question open. Whether a reviewer who approves thirty rows in eight seconds is exercising oversight is not a UI question with a UI answer, and the paper above is the beginning of that literature rather than a resolution of it.

What is worth returning to is the altitudes reference, linked below, which holds the whole disclosure model in one page: which fact belongs at which altitude, who reads it, and what breaks when the wrong thing is put there. The diff is the altitude a reviewer acts from, and it only works because the three below it exist.

Then there is the leave-behind checklist in ARTIFACT.md, which is the real end of this course. Nothing on it is ticked by reading. The last item — that <AgentTrace /> renders a payload from a runtime it was not written against, and fails by naming the missing field rather than by rendering an empty tree — is the one that turns four modules of reading into something an interviewer can open.