Traces as the observability substrate
A trace is a tree of runs with inputs, outputs, timings and errors on every node — the same data an agent UI needs, already collected, which makes the tracing backend the fastest way to learn an unfamiliar graph.
You have three days before a design review on a product whose agent somebody else built. Two options. Read the repo, which means reading a graph, its subgraphs, its prompt templates and whatever the tool layer turned out to be. Or open the tracing project, pull ten real runs, and read what the thing actually did on real inputs.
This course’s verdict, and it is this course’s verdict rather than anybody’s documented recommendation: start with the traces. Source tells you what can happen. A trace tells you what does happen, how often, in what order, and where it fails. For deriving a UI, the second question is the one that pays.
Two definitions, and why they are the whole model
LangSmith’s observability documentation defines the two nouns precisely. A run is “a single unit of work executed by an agent, such as calling an LLM, formatting a prompt, or retrieving documents”, and a trace is “a collection of runs for a single operation.” Both definitions come from the same page, fetched for this course on 2026-09-02, which also offers the sentence that makes this portable: you can think of a run as a span.
That sentence is the reason this lesson is in the not-being-dogmatic module rather than in a LangSmith module. If a portfolio company traces into Datadog, Honeycomb, or a plain OpenTelemetry collector, the vocabulary still lands. A trace is a tree of spans. An agent trace is a tree of spans that happens to have model calls and tool calls in it. Nothing about the reading skill is tied to the vendor.
What one run actually carries
The run data format documents a wide record. The fields that matter for building a surface:
| Field | What it gives your UI |
|---|---|
inputs / outputs | The real payload shapes, on real data. This is your API contract, observed rather than described. |
run_type | What kind of work it was. Seven documented values: chain, llm, embedding, prompt, tool, retriever, parser. |
start_time / end_time | How long a step takes in production, which decides whether it needs a progress surface at all. |
error / status | The failure states your design has to have a screen for, with real frequencies attached. |
tags | Whatever the team chose to label runs with. Often the fastest guide to how they think about their own graph. |
Token and cost fields ride along too. Notice what that list already gives you: the same data an agent UI needs, collected before you arrived, for the exact reason that somebody wanted to debug it.
Two fields that are not there
Both of these are things a designer reasonably assumes and then builds a promise on.
There is no top-level metadata. The page is explicit: no top-level metadata field exists, and metadata is stored inside extra. So a filter you sketch as “show me all runs where metadata.tenant is X” is reaching for a path that does not exist at that level, and whoever implements it has to know where to look.
There is no latency. The page states that no latency field is stored, and that latency is derivable but not pre-computed. You get it by subtracting the timestamps. The one related field that is stored is first_token_time, and it is scoped: runs with run_type="llm" and streaming enabled. That single field is the honest source for “time to first token” on a streaming surface, and it does not exist for anything else in the tree.
Where people get burned
“Sort the run list by slowest step” sounds like a filter and is a computation. Someone has to derive it from start_time and end_time, decide what to do with runs that have no end_time because they are still going or died, and agree where that derivation runs. This is exactly the class of request that looks free in a design file and costs a sprint. Ask which fields are stored before you draw a column.
How a flat list becomes a tree
A trace arrives as a set of runs, not as a nested object. Two fields reassemble it. dotted_order is documented as a sortable key that fully specifies a run’s location within the tracing hierarchy, in the form <run_start_time>Z<run_id>.<child_start_time>Z<child_id>, and trace_id equals the first UUID in that dotted order.
// Sorting by dotted_order gives you the tree in reading order,
// because the key is start-time-prefixed at every level.
const ordered = runs.sort((a, b) =>
a.dotted_order.localeCompare(b.dotted_order)
);Two consequences worth carrying into a design conversation. Sorting a flat run list lexicographically by dotted_order yields the tree in the order a human would read it, so a nested view is a rendering problem and not a data problem. And because the key is start-time prefixed at every level, sibling ordering is by start time — which is exactly what you want for reading a fanned-out superstep, and exactly not what you want if you wanted them ordered by completion.
Reading a graph you have never seen, in one sitting
The method, which is this course’s own and is not a documented recommendation:
- Pull ten completed traces and ten failed ones. The failed ones are worth more.
- Census the
run_typevalues. A tree that is nearly allchainwith twollmruns is a workflow with a model in it. A tree with manytoolruns under onellmrun is an agent loop. Those want different UIs, and you now know which one you have without opening the repo. - Read the
inputsandoutputsof the two or three deepest runs. That is the state contract, observed. - Derive durations from the timestamps and find the steps over a few seconds. Those, and only those, need progress surfaces.
- Read every
erroryou can find. Each distinct one is a screen somebody has to design, and the count tells you which to design first.
Then, and only then, open the source — with a list of specific questions instead of a cold read.
One practical note on getting access. Current LangSmith documentation uses LANGSMITH_TRACING and LANGSMITH_API_KEY. If you inherit a repo using the older LANGCHAIN_-prefixed names, do not assume they are dead: no page this course fetched states that either set is deprecated. Use the current names in new code and leave the old ones alone.
Check your recall
Answer from memory — no scrolling back.
Retrieval check
Day one at a portfolio company. You have read access to their tracing project and to the repo, and a design review on Thursday. What do you do first, and what do you have by lunch?
Check your answer
Traces first, source second. Ten successful runs and every failed run you can find.
By lunch you should have: the shape of the tree (which tells you workflow versus agent loop), the real input and output payloads at the deepest nodes (which is the state contract as observed rather than as described), the steps slow enough to need a progress surface (derived from the timestamps, since latency is not stored), and the distinct error states with rough frequencies.
That is most of a surface inventory, produced without understanding the code. The source read that follows is then targeted: you are looking for why a specific branch fires, not for what the system is.
The claim that this is faster than reading source is this course’s, not LangChain’s. Test it on your own graph before you say it in an interview.
Hands on
Trace your own graph, then read it as a stranger
Done when: ARTIFACT.md’s module 4 section contains a run-type census for one real trace of your parser graph, the derived duration of every run in it, and a short list of the surfaces that census implies — written as if you had never seen the code.
- Turn tracing on for the graph you built in the earlier modules, using the current
LANGSMITH_environment variable names, and run it end to end at least twice: once clean, once with a deliberate failure inside a node. - Export or copy one full trace. Count the runs by
run_typeand paste that census into the artifact. Say which of the two shapes it is, workflow or agent loop, and what that implies for the top-level UI. - Derive the duration of each run from
start_timeandend_time. Do not look for a latency field. Mark every run over three seconds as needing its own progress surface. - Find the failed run. Record its
errorandstatusverbatim, and write the one sentence a user would need to see instead of that string. - Now the real exercise: write the surface list from the trace alone, without consulting your own graph definition. Then open the definition and mark everything the trace did not reveal. That gap is the honest limit of this technique, and knowing it is what stops you overselling it.
What this does not cover
A trace shows the model calling a tool and shows what came back. It says nothing about who was supposed to approve that call, or where the standard for tool exposure says the confirmation belongs. That is the mcp-as-the-tool-boundary lesson, and it is the one place in this course where a UI requirement is written down in a protocol specification rather than argued for.
Nothing here covers the resume path either — a trace records that a run paused, but the durable state that lets it be picked up belongs to the checkpointer, which the checkpointers-and-threads lesson covered. And the repeatable pass that turns traces, source and interrupts into a single deliverable is the reading-an-unfamiliar-stack lesson, which closes the course.
Read this next — primary source
Run (span) data formatLangChain — docs.langchain.com, LangSmith docs, fetched 2026-09-05. Vendor documenting a product it sells
This lesson takes the run schema and two of its corrections: there is no top-level metadata field, and latency is not stored. Read the whole page, because the value here is knowing which fields exist before you promise a screen that depends on one. It stops well short of telling you what to build; every design conclusion in this lesson is drawn by this course from that schema, not stated by LangChain.
Stuck, curious, or think this lesson is wrong? Ask your teaching agent. The lessons are the scaffold; the conversation is where the learning gets unstuck.