Source-of-claim highlighting
Highlighting the exact span of source text a sentence rests on turns verification from reading a document into checking a match, which is the difference between a citation being decorative and being used.
The preview from the last lesson opens next to the sentence and shows the passage. Now count the words in it. A retrieved chunk is commonly two hundred to a thousand words, and the sentence in front of the reviewer rests on maybe fifteen of them. You have replaced a document search with a paragraph search. Better, and not finished.
The obvious next move is to highlight the span. Show the fifteen words, dimmed against the rest, so the reviewer’s eye goes to the part that is supposed to carry the claim.
Before building that, this lesson owes you a disclosure, because it is the one place in this module where the course is arguing rather than reporting.
The argument, stated so it can be attacked
Reading a passage to decide whether it supports a sentence is an open task. The reviewer has to hold the claim in mind, scan for anything related, judge relevance as they go, and decide when they have read enough to stop. There is no completion signal. Under time pressure, the task with no completion signal is the one that gets skipped.
Matching two spans is a closed task. Here is the sentence, here are the words it is supposed to rest on, do they say the same thing. It has an end. The reviewer knows when they are done, and they know what a failure looks like.
The course’s position is that converting the first task into the second is what makes checking cheap enough to happen at all, and that this is a larger effect than any styling decision about the anchor. That is the claim. It is not measured.
The reason to be suspicious of it
There is a result that cuts directly against building this feature and feeling good about it. Bansal and colleagues, at CHI 2021, found that “explanations increased the chance that humans will accept the AI’s recommendation, regardless of its correctness.”
A highlight is an explanation. It is a visually confident assertion that these particular words are the reason. If the span is wrong, the highlight does not become less persuasive; it becomes a wrong claim delivered with the same visual authority as a right one. The mechanism that makes highlighting valuable when the span is correct is the same mechanism that makes it dangerous when it is not.
That gives you the actual design requirement, and it does not depend on whether the course’s argument above is true: the highlight must say where the span came from, because the span has different trustworthiness depending on who produced it.
Three places a span comes from, and what each one licenses
A highlighted range is a fact about your pipeline, not a fact about the world. There are three common producers and they are not interchangeable.
| Producer | What it is actually telling you | The sentence the UI may write |
|---|---|---|
| The retriever | Which chunk was placed in the model’s context for this turn | “This is the passage the model was given.” Says nothing about support. |
| The model, returning offsets | Where the model asserts the support is. Generated text like any other output. | “The model points here.” Needs the same scepticism as the answer itself. |
| A separate matcher run after the fact | What one checker scored as the best-supporting span, on its own criteria | “Matched by X on date.” Named checker, or the badge means nothing. |
The retriever case is the one most often mislabelled, because it is the one most often available. Chunk retrieval is a similarity operation. It tells you a passage was close enough to the query to be worth including. It cannot tell you that the passage supports a sentence the model wrote afterwards, and a UI that renders retrieval provenance with the words “source of this claim” has upgraded a similarity score into an entailment claim without anything performing the entailment.
The model-offsets case has a failure that is easy to miss in development. A model asked for character offsets will produce character offsets. They may not point at anything. Off-by-a-few is common, confidently wrong is possible, and a highlight rendered from unvalidated offsets can land on adjacent text that happens to read plausibly, which is the worst available outcome: a wrong span that survives a reviewer’s glance.
Where people get burned
Validate every span against the passage before you render it. If the offsets do not resolve, or the quoted text is not found in the source, the component shows the passage with no highlight and says the span could not be located. It does not silently fall back to highlighting the first sentence, and it does not fuzzy-match its way to something nearby.
The reason is the same one that runs through the whole course. A failed lookup rendered as a successful one is not a cosmetic bug. It is the component asserting something the data never said, in the one place a reviewer has no way to check it.
Highlighting is a claim about the sentence too
One more asymmetry, and it is the one most implementations miss. A citation attaches to a sentence, but the support usually does not cover the whole sentence. Consider:
The property was surveyed in 2019 and the boundary has not moved since.
The source may establish the survey date and say nothing at all about the boundary. One highlighted span, attached to that whole sentence, tells the reviewer the sentence is supported. Half of it is.
This course’s own answer, again labelled as such: highlighting is a two-ended relationship, and a component that only marks the source end is doing half the job. If the pipeline can produce which part of the claim a span supports, mark both ends. If it cannot, the honest fallback is to keep the citation attached to the sentence and avoid any visual language — a full-sentence underline, a tick, a green wash — that reads as the whole sentence being covered.
Retrieval check
Your retrieval pipeline returns the chunk it fetched, and nothing marks a span inside it. What do you render, and what do you refuse to render?
Check your answer
Render the chunk, in full, labelled as what it is: the passage this run retrieved and put in front of the model. That is a true and useful statement, and it is strictly better than a URL.
Refuse the highlight, and refuse the label. The words “source of this claim” assert a relationship between a span and a sentence that nothing in your pipeline computed. You have retrieval provenance, which answers “what did the model read?” The reviewer is asking “does this back up that?” Those are different questions, and the gap between them is exactly where an unsupported citation lives.
The productive version of this constraint is that it names the missing capability precisely. “We have retrieval provenance and no claim-level provenance” is a sentence that gets a pipeline changed. “The citations feel weak” is not.
Hands on
Decide what your highlight is allowed to say
Done when: ARTIFACT.md’s “Source-span highlighting” field names which of the three producers your pipeline has, the exact label the UI writes, and the rendering for a span that fails to resolve.
- Name the producer. Retriever, model offsets, or a separate matcher. If the answer is “the retriever, and we were going to call it the source of the claim,” you have found the finding of this exercise.
- Write the label as a literal string, the way it will appear on screen. Then read it back and ask whether your producer earns every word of it. “Supports” and “source of” are the two words that most often will not survive this.
- Write the validation rule: how the component confirms a span resolves inside the passage before drawing anything, and what it renders when it does not. State it as a rule, not an intention.
- Take a sentence from your own run that makes two assertions at once, the way the survey example does. Decide what the component does about the half the source does not cover, and write that decision down even if the decision is to do nothing yet.
- Write one line naming what would have to be true for the highlight to be making things worse rather than better, given the Bansal result. You are keeping the counterargument in the artifact on purpose. It is the thing an interviewer will ask about, and having written it already is the whole point.
- Bring the label string and the counterargument line into the chat. Those two are where the disagreement will be, and the disagreement is worth more than the markup.
What this does not cover
Everything above assumed the passage does support the sentence and the only problem is finding the part that does. That assumption has been measured across production systems, and it fails often enough to need its own design treatment. The unsupported-citation lesson, next, takes the case where the URL is real, the passage is real, and the page simply does not say what the sentence claims.
It also stayed away from how sure the system is about any of this. A span that a matcher scored, a retrieval that returned a similarity number, a model that offered a percentage — those are confidence signals, and what a component is allowed to do with one is the confidence lesson, at the end of this module.
Read this next — primary source
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team PerformanceGagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, Daniel S. Weld, CHI 2021; arXiv June 2020 — free
This is deliberately not a source that supports the lesson. It is the paper most likely to prove the lesson wrong, and it is worth reading in full for that reason. The finding — that explanations increased the chance humans accept the AI’s recommendation regardless of its correctness — applies to every explanation feature you will ever ship, and a highlight is an explanation. Read the experimental design closely, particularly how they separate an explanation that improves the team’s accuracy from one that only improves acceptance. That distinction is the test any highlighting feature has to pass, and this lesson cannot tell you whether the one you build passes it.
Stuck, curious, or think this lesson is wrong? Ask your teaching agent. The lessons are the scaffold; the conversation is where the learning gets unstuck.