Loading
Every factual claim in this course traces to something here, grouped by the module it serves. Two of these groups serve modules that have not shipped yet — the sources were verified in the same pass, and are listed now so the reading can run ahead of the lessons.
The trace model, the vocabulary layered on top of it, and the actual emission shapes of real runtimes.
WhySource of the span anatomy the span-model lesson teaches: name, immutable SpanContext, a parent “in the form of a Span, SpanContext, or null”, SpanKind, start and end timestamps, attributes, links, timestamped events, status. Also the sentence that reshapes any status rendering — “Instrumentation Libraries SHOULD NOT set the status code to Ok… SHOULD leave the status code as Unset unless there is an error” — so a successful step is Unset, not Ok, and the three codes form a total order Ok > Error > Unset. Description MUST only be used with Error. Attribute values are constrained to string, boolean, float, integer or arrays of those, with no object type, which is why structured content arrives as a JSON string. Primary source for the span-model lesson.
WhyThe readable counterpart to the specification. Lists the span fields including “Parent span ID (empty for root spans)”, which is the whole tree mechanism, and gives the attribute-versus-event heuristic the lesson uses: if the timestamp at which something happened is meaningful, make it a span event; if not, make it an attribute. Also defines links as associating spans across the same or different traces, “implying a causal relationship”.
WhyThe propagation format: traceparent as version, 32-hex trace-id, 16-hex parent-id and 8-bit trace-flags, with the worked example 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01. Two details the lesson leans on: the field called parent-id is, in the spec’s own words, the span id under another name, and trace-flags are “recommendations given by the caller rather than strict rules” — the basis for treating a trace as sampled and advisory rather than complete.
WhyCited only to date it. Level 2 is a Candidate Recommendation Draft rather than a Recommendation, so the stable propagation story remains the 2021 document even though the OpenTelemetry tracing API already references Level 2 for its Random trace flag.
WhyWhere the generative-AI conventions actually live now. The old location, opentelemetry.io/docs/specs/semconv/gen-ai/, still resolves but now contains only a “Moved” notice stating the page “is no longer maintained in this repository” — a stub with no attribute tables. Two consequences the conventions lesson turns into practice: there is no rendered documentation site for the moved content, and the repository carries no tagged release, so any URL you can cite points at a moving main branch. Cite a commit, not a branch, when it has to stay true.
WhyThe attribute tables the conventions lesson works from: gen_ai.operation.name and its eighteen well-known values, gen_ai.provider.name, gen_ai.request.model, gen_ai.usage.input_tokens / output_tokens, and the tool-span set gen_ai.tool.name, gen_ai.tool.call.id, gen_ai.tool.type, gen_ai.tool.call.arguments, gen_ai.tool.call.result. Also the span-name formulas ({operation} {model}, execute_tool {tool}, invoke_agent {agent}) and the “Capturing instructions, inputs, and outputs” section, which states message content SHOULD NOT be captured by default, marks gen_ai.system_instructions / input.messages / output.messages Opt-In, and recommends external storage with span references for production. Primary source for the conventions lesson.
WhyOne of the few GenAI pages still rendered on opentelemetry.io, and the evidence base for “these names move”: gen_ai.system is deprecated in favour of gen_ai.provider.name, gen_ai.usage.prompt_tokens and completion_tokens in favour of input_tokens and output_tokens, and gen_ai.prompt and gen_ai.completion are “Removed, no replacement at this time.” Even values churned — the deprecated gen_ai.system listed xAI as xai, while the current gen_ai.provider.name uses x_ai.
WhyThe source for the course’s most load-bearing negative finding: there is no normative rule that a tool execution nests under the model call that requested it. On a chat → tool → chat sequence the document says “the relationship between below spans depends on how user application code is written,” and that the spans are likely to be siblings under an encompassing span. The tree shape in a captured payload is therefore an artifact of that program, not a semantic fact.
WhyA complete emission catalogue for one runtime, and the sharpest single illustration that attribute naming is unsettled: the page documents two live integrations at once. The recommended one emits GenAI conventions — spans invoke_agent {modelId}, chat {modelId}, execute_tool {toolName}, with gen_ai.usage.input_tokens / output_tokens. The other, which Vercel itself labels “the legacy format”, emits ai.generateText, ai.generateText.doGenerate, ai.toolCall and names the same integers ai.usage.promptTokens and ai.usage.completionTokens. Note also that Vercel’s root span name interpolates the model id where the convention prescribes the agent name — even a stated follower diverges on the name, which is why the course keys off gen_ai.operation.name instead. Primary source for the end-to-end lesson.
WhyThe protocol-level shape of a tool-calling loop: stop_reason "tool_use", a tool_use block with exactly id, name and input, and a tool_result block with tool_use_id, content and an optional is_error boolean, returned in a message with role user. Also the ordering constraints — results must immediately follow their calls, and tool_result blocks must come first in the content array with text after them — and the design statement that, unlike APIs using a separate tool or function role, tools are integrated into the ordinary user and assistant message structure.
WhyThe seven documented stop_reason values: end_turn, max_tokens, stop_sequence, tool_use, pause_turn, refusal and model_context_window_exceeded. Cited to show that “the loop ended” is seven different states, three of which are failures a trace panel should not render as a green check.
WhyA second, incompatible carving of the same domain, used in the first lesson to show there is no single shape called “an agent run.” A Run has exactly seven run_type values — chain, llm, tool, retriever, embedding, prompt, parser — and notably no agent type at all. It documents start_time and end_time but no duration field, the concrete example the lesson uses for a derived rather than emitted value. Its dotted_order encodes depth and chronology in one lexicographically sortable string, so the trace is identified by its root run’s id rather than by a separate trace entity.
WhyThe mapping tables are the document. LangSmith translates gen_ai.*, OpenInference, TraceLoop, generic llm.* and Logfire attributes into its own langsmith.* fields, row by row — a translation layer a commercial vendor had to build and maintain, which is the strongest available evidence for the first lesson’s claim that the vocabularies genuinely differ. Also confirms the deprecation direction independently: gen_ai.usage.prompt_tokens and completion_tokens are marked deprecated in the vendor’s own table. Primary source for the view-comes-second lesson.
WhyA third carving, cited for contrast. Requires openinference.span.kind on every span, with a longer list than LangSmith’s seven run types — LLM, EMBEDDING, CHAIN, RETRIEVER, RERANKER, TOOL, AGENT, GUARDRAIL, EVALUATOR, PROMPT. Note the spec text itself does not claim vendor-neutrality; that framing comes from Arize’s marketing pages rather than the specification.
WhyNamed for completeness as a fourth vocabulary in the mapping tables above. An open-source project built on OpenTelemetry with its own GenAI conventions and many export targets. Its license and maintainer list could not be verified from the documentation page, so it is listed here as a name that appears in translation tables rather than as a recommendation.
WhyA fifth model, structurally different again: a Trace is “a single end-to-end operation of a workflow” with workflow_name, trace_id, group_id and metadata, and spans carry typed span_data classes rather than attribute bags — agent_span, generation_span, function_span, guardrail_span, handoff_span among them. Useful as the contrast case: typed span data versus OpenTelemetry’s flat attributes, converging on the same three shapes of agent, model call and tool call.
How much to explain, to whom, and the usability research on layering it — two design-guidance publishers with commercial positions of their own, and one finding the course’s four altitudes have to argue with rather than ignore.
WhyPrimary source for the four-altitudes lesson, and the one finding that lesson has to meet head-on: “designs that go beyond 2 disclosure levels typically have low usability because users often get lost when moving between the levels.” Four altitudes is twice that. The resolution the lesson offers — two disclosures on the product surface plus a separate two-level inspector entered through a door — is the course’s own construct, not Nielsen’s, and the lesson says so. Also the source of the framing the course borrows wholesale: the difficulty of progressive disclosure sits entirely in deciding which features are initial and which are secondary.
WhyQuoted in the four-altitudes lesson for “Because these technical roles bring different goals, expertise, and contexts, explainability cannot be one-size-fits-all,” and for the split it draws between a governance lead wanting global explanations, a builder wanting local and interactive ones, and a domain expert wanting local, static ones with somewhere to give feedback. The direct evidence for the altitudes test: name the reader, then name what they do next.
WhyThe disclosure budget the four-altitudes lesson runs on: “The best approach is not to attempt to explain everything – just the aspects that impact user trust and decision-making,” with progressive disclosure named explicitly as the companion technique that keeps the rest reachable rather than absent. The chapter also frames the goal as calibrated rather than maximal trust. Its confidence material — the heading “Determine if you should show confidence” and the warning that a misleadingly high confidence may cause users to blindly accept a result — belongs to the citations and confidence module and is only pointed at from here.
WhyCited in the honest-one-liner lesson for “Be up-front about what your product can and can’t do the first time the user interacts with it.” The collapsed summary is first contact for every run, which makes that instruction a constraint on one line of text rather than a general principle about onboarding.
What the research says about showing sources and numbers to people making decisions — and, on one lesson, the fact that it says nothing at all.
WhyPrimary source for the inline-anchors lesson. Three findings the citations module rests on: people rarely click citation links — illustrated by a participant who said they would check the source and then clicked none — so sources should sit directly adjacent to the specific claims they support rather than buried in response text and be styled distinctly; chain-of-thought explanations “are often unfaithful to the model’s actual computation”, being post-hoc rationalisations that can omit influencing factors or adjust to justify a wrong answer; and disclaimers should be plainly worded and paired with an action rather than buried in footers. The article does not discuss progressive disclosure, and it does not test or recommend a hover preview — the anchor/preview/jump-to-source pattern the lesson builds is the course’s own response to the click-rate finding, not something NN/g prescribes.
WhyPrimary source for the unsupported-citation lesson, and the measured version of the failure it is named after. Across Bing Chat, NeevaAI, Perplexity.ai and YouChat, on average “a mere 51.5% of generated sentences are fully supported by citations and only 74.5% of citations support their associated sentence.” Its two-axis vocabulary — citation recall (is every statement supported) versus citation precision (does every citation support its statement) — is the framing the lesson uses, because they are separate failures needing separate UI. The figures measure four named products as they existed when the paper was written; the lesson cites them as evidence that unsupported citations are a normal design case, never as a rate that applies to any component built on this course.
WhyPrimary source for the source-of-claim-highlighting lesson, and cited there against the lesson’s own argument rather than for it: “explanations increased the chance that humans will accept the AI’s recommendation, regardless of its correctness.” A highlight is an explanation, so the same mechanism that makes a correct span useful makes an incorrect one persuasive. Also carried into the confidence lesson, where it sits beside Zhang, Liao & Bellamy as the pair of results showing that transparency features move trust far more reliably than they move accuracy.
WhyCited in the confidence lesson for the limit on what a confidence display can achieve: a confidence score “can help calibrate people’s trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making, which may also depend on whether the human can bring in enough unique knowledge to complement the AI’s errors.” Humans and AI had comparable solo performance in the study, which the lesson states when borrowing the result. Note the arXiv comments field says CHI 2020 while the DOI resolves to the FAccT proceedings; cite it as FAccT.
WhyCited in the confidence lesson for the definition only — calibration as “predicting probability estimates representative of the true correctness likelihood” — plus the finding that “modern neural networks, unlike those from a decade ago, are poorly calibrated.” It studies image and document classification networks, not language models, and the lesson says so on the page. It establishes that a stated confidence and an actual correctness rate are separable quantities, and that checking the difference requires labelled outcomes at volume. It is never evidence about how any language model is calibrated today.
WhyPrimary source for the confidence lesson, and the language-model-specific evidence for refusing to render a stated percentage as a measurement: “LLMs, when verbalizing their confidence, tend to be overconfident, potentially imitating human patterns of expressing confidence,” tested across five models and five reasoning datasets. The scope is load-bearing and the lesson keeps it — the finding is about verbalised, self-reported confidence, not about every mechanism that can be called confidence. The same paper reports that white-box methods reading internal signals still beat verbalised confidence, though “the gap is narrow, e.g., 0.522 to 0.605 in AUROC”, which is close enough to the 0.5 of a coin flip to license ordering a review queue and nothing proportional.
Where the numbers come from and how fast they go stale. Three of these five are vendors stating prices for their own products, every price carries the date it was read, and the one paper here argues against the lesson that cites it.
WhyPrimary source for the per-step cost lesson, and the reason that lesson prints no rate card. On 2026-09-05 the page carried this note: “The $2/$10 per million input/output token pricing for Claude Sonnet 5, announced at launch as introductory pricing through August 31, 2026, is now the standard price. The previously scheduled increase to $3/$15 per million input/output tokens on September 1, 2026 will not occur.” An announced, dated price change was reversed within days of taking effect, which is the strongest available argument for storing a rate with its fetch date rather than as a literal. Two structural facts outlive the numbers: a model has several rates at once (input, output, cache read, cache write at more than one lifetime, batch on its own discount), and — recorded from this page on 2026-09-02 — Claude 4.7 and later use a newer tokenizer producing roughly 30% more tokens for the same text, so a lower per-token price does not mechanically mean a cheaper run. Note that claude.com/pricing carries consumer plan tiers only and no API token prices.
WhyThe usage object in literal form — input_tokens, output_tokens, cache_creation_input_tokens, cache_read_input_tokens — plus the identity the per-step cost lesson is built on: the true input total is cache_read + cache_creation + input_tokens, because input_tokens counts only the tokens after the last cache breakpoint. Multiplying gen_ai.usage.input_tokens by a base input rate on a cached run is wrong in both terms at once, and the two errors point in opposite directions, so the result stays plausible.
WhyCited in the per-step cost lesson for structure rather than for any single figure: input, cached input and output are listed as separate rates per model, which is why a cost function with one rate per model cannot express the run it is pricing. The roster of models on the page changed between this course’s two fetches three days apart, which is the evidence behind the lesson’s instruction not to build a component around “the current flagship”. Note the URL — openai.com/api/pricing/ returned HTTP 403 to automated fetching, so this developers.openai.com page is the one actually verified.
WhyPrimary source for the roll-up lesson. Instruments and their types, which is the part that carries the argument: gen_ai.client.token.usage (Histogram, {token}), gen_ai.client.operation.duration (Histogram, s), gen_ai.invoke_agent.duration (Histogram, s), gen_ai.execute_tool.duration (Histogram, s), gen_ai.invoke_agent.inference_calls (Counter, {call}), gen_ai.invoke_agent.tool_calls (Counter, {call}). Histograms and counters are aggregates by construction — neither can be read off one run nor unwound back into one — which is the specification-level evidence that a population question is a different data shape from a trace rather than a zoomed-out view of one. Verified at instrument-name and instrument-type level; the per-metric attribute tables have not been checked in detail.
WhyPrimary source for the diff-view lesson, and it argues against that lesson’s starting instinct rather than for it. Studying review activity on AI-generated pull requests in the AIDev dataset against human-authored pull requests in the same repositories, the authors report that “most AI-generated PRs receive no review and, when reviewed, are largely dominated by AI agents rather than humans”, with review of agent-authored changes more often taking “the form of automation-mediated interaction”, and they caution against reading review metrics as a proxy for human oversight. It is cited to withdraw a claim: no source was found establishing that a diff lets a reviewer approve or reject safely without reading the reasoning, and this is the closest empirical work to that claim. Scope caveat: it measures GitHub pull requests, not agent review surfaces in general.
Every claim on these pages links to its source. If a source looks wrong or out of date, check the resource list and tell your teaching agent — the course is meant to be corrected.