The roll-up an operating partner actually wants

The executive view is not a smaller trace — it answers a different question, about a population of runs rather than one run, built on a different OpenTelemetry signal.

Someone who has never opened your trace panel is about to ask you a question, and none of the work so far answers it. Not “what did this run do,” but “what do these runs cost us, how often do they fail, and is that getting worse.”

The instinct is to reach for the component you already have and zoom out. That instinct is wrong for a reason that is visible in the specification rather than in taste: a trace and a roll-up are built on two different signals, because they answer two different kinds of question, and OpenTelemetry keeps them apart on purpose.

Two signals, not one view at two zoom levels

OpenTelemetry defines traces and metrics as separate signal types. The GenAI conventions follow that split with a separate document for each, and the metrics one defines instruments rather than spans. As read on 2026-09-05 from main, the metrics conventions include:

Instrument	Type	Unit
`gen_ai.client.token.usage`	Histogram	`{token}`
`gen_ai.client.operation.duration`	Histogram	`s`
`gen_ai.invoke_agent.duration`	Histogram	`s`
`gen_ai.execute_tool.duration`	Histogram	`s`
`gen_ai.invoke_agent.inference_calls`	Counter	`{call}`
`gen_ai.invoke_agent.tool_calls`	Counter	`{call}`

Every one of those is marked Status: Development, in a repository with no tagged release, which is the standing warning of this course and applies here exactly as it applied to the span attributes.

The types are the point. A histogram is a distribution over many observations. A counter is a monotonic total. Neither is a thing you can read off one run, and neither can be unwound back into one run. That is not an accident of the file layout — it is what makes them the right substrate for a population question and the wrong one for “what happened in this particular execution.”

Why the same component cannot do both

This part is the course’s own position rather than something the specification says, and it is worth stating as such. No source was found measuring how readable existing cost dashboards are, so treat what follows as an argument, not a finding.

The argument: a trace renderer is built around identity. Every row is a specific span with a specific id, and the interactions — expand, inspect, jump to the tool call — all assume that clicking a thing takes you to that thing. A roll-up row is not a thing. It is a bucket. Clicking it cannot take you to a span, because it stands for hundreds of them. Reuse the trace component and you inherit an affordance that cannot be honoured, and the usual repair is to make the row link to one arbitrary example run, which quietly turns a distribution into an anecdote.

The second inheritance is worse. A trace panel is complete by construction: it shows every span it was given. A roll-up is never complete, because it covers a window and a population, and both of those are choices. A component that never had to render a denominator will not grow one.

The three things a roll-up number needs

Every figure on the executive view carries three pieces of context, and the number is unreadable without them:

A denominator. Over how many runs. A failure rate without a run count is a rumour.
A window. Between which two timestamps. “This month” is not a window until the panel says which month and whether the current one is partial.
A provenance date for anything in currency. Straight from the per-step cost lesson: a dollar total is tokens multiplied by a rate card read on a particular day. Roll one thousand runs up and the rate-card date does not stop mattering, it starts mattering more, because the window may straddle a price change.

Then one more, which is a distribution rather than a caption: report a shape, not an average. The conventions define duration and token usage as histograms rather than gauges, and a histogram is exactly what a median and a high percentile need. Those two answer a question the mean cannot: is this slow for everyone, or fast for most and catastrophic for a few? The two situations have entirely different fixes and identical averages.

The count that is a lower bound

If you compute counts by aggregating stored traces rather than by reading instruments, remember what module one established: sampling is built into the trace model, and W3C Trace Context calls trace flags “recommendations given by the caller rather than strict rules”. A run that was not sampled left no trace to count. So a trace-derived run count is a lower bound wearing the typography of a total, and the roll-up has to say so or stop reporting it.

What actually goes on the page

A defensible first roll-up is smaller than people expect. One row per agent or workflow, and per row: run count over the stated window, completion rate, the duration distribution as median and a high percentile, inference calls and tool calls per run, and a cost band with its rate-card date. Six columns, every one of them either emitted or derived from something emitted, and none of them a single number pretending to summarise a distribution.

What does not go on it: a reasoning excerpt, a step tree, a best-or-worst example run presented as typical. Those belong to the altitudes below this one, and the way down should be a filter that produces a list of runs, not a link that produces one.

Retrieval check

Your roll-up says the average run costs a certain amount. Finance builds next quarter’s budget on it and the actual bill comes in far higher. Nothing in your query was wrong. What happened?

Check your answer

The first thing to check is whether the mean was computed over a right-skewed distribution while the budget needed the tail. A loop that terminates on a condition can produce that shape: most runs finish in a few model calls and a few keep going. Nobody has measured that distribution for your runs, which is the point — the mean cannot tell you whether it holds, and if it does hold, the mean sits below the runs generating most of the spend, so a per-run mean multiplied by an expected volume underestimates.

The second candidate is the denominator. If failed runs were excluded from the average but still billed — and a run that burned fifteen model calls before giving up is expensive precisely because it failed — then the population you averaged is not the population that gets invoiced. Both faults are invisible in a single number, which is why the conventions expose duration and token usage as histograms rather than as gauges.

Check your recall

Answer from memory — no scrolling back.

1. You are asked for the 95th percentile of agent run duration. What is the honest way to get it from a single stored trace?
Sort the spans by duration and take the 95th percentile of the step timesTake the root span duration and add a margin for varianceYou cannot — a percentile is a property of a population, and one trace is one observationUse the longest child span as a proxy for the tail
2. Your roll-up shows a total spend for the last quarter. Which caption keeps it honest?
The total, with a note that prices may varyThe total, the run count it covers, the exact window, and the date the rate card was fetchedThe total, plus a link to a representative runThe total, rounded, so nobody reads more precision into it than it has

Hands on

Write the population question down before you draw anything

Done when: ARTIFACT.md’s Roll-up field states one population question in a single sentence, names the instrument or derivation each column comes from, and records the denominator, window and rate-card date the view will display — with no column left whose source you cannot name.

Write the question. One sentence, in the words of the person who would ask it, with a window in it. If you cannot get a window into the sentence, you are still describing a trace view.
List the columns that answer it. For each one, write where the number comes from: an instrument name from the metrics conventions, or a derivation over stored runs. Any column you cannot source is a column you were going to invent.
For every derived column, write the denominator and the window explicitly, as strings the component will render. Do this before the markup exists, because a caption added afterwards gets cut for space.
Replace every mean with a median and one high percentile. Then look at your captured payload and check whether you have enough runs to compute either. If you have one run, record that the roll-up is currently unpopulated rather than shipping a distribution over a sample of one.
Decide what happens when a reader clicks a row, and write it down. If the answer is “opens a run,” say which run and why that is not misleading. If you cannot defend it, make the row filter a list instead.
Bring the question sentence and the column-to-source table into the chat. The rows worth arguing about are the ones sourced to a derivation rather than an instrument, because those are the ones that quietly depend on which runs happened to be stored.

What this does not cover

This lesson stayed on aggregates: the shape of a population question and the signal that answers it. It does not build the collector side. How instruments get recorded, exported and stored is operations work, and this course renders telemetry rather than running it — that boundary was set in the mission and it holds here.

It also says nothing about the one artifact an executive view cannot produce and a reviewer most needs: what the run actually changed. Cost and duration describe what a run consumed. The closing lesson of the course, on the diff view, is about what it did.