Retrying mid-stream
A stream that dies at token 400 leaves partial state on screen and partial state on the server, and the retry that looks correct in a demo is the one that silently duplicates half an answer.
The stream dies at token four hundred. Not at the start, where every error handler you have ever written expects it. Four hundred tokens in, with three quarters of an award-transfer recommendation on screen, a tool result already rendered, and possibly a database row you opened when the first delta arrived.
The user sees a truncated answer and a retry button, because you added one. They press it. What happens next is the subject of this lesson, and the failure mode is specific: it works perfectly every time you test it, because you test it by killing the stream at token zero.
Two partial states, not one
A dead stream leaves state in two places and people reason about only the first.
On screen: four hundred tokens of assistant message, rendered, read, and now permanent in the user’s memory of the conversation whatever you do with the pixels.
On the server: a generation that, under at least one vendor’s default wiring, is still running — the stop-button lesson established that a disconnect is not a cancellation unless somebody wired it to be. Plus whatever you persisted as the deltas arrived, plus whatever a tool call did to the outside world before the connection dropped.
A retry that only reconciles the first one is the retry that duplicates. The classic shape: the client appends new deltas to the same message node, the second stream starts from the beginning, and the user reads four hundred tokens followed by the same four hundred tokens again, joined at a sentence boundary convincing enough that nobody notices for a week.
The mechanism you are looking for does not exist
Here is where most people reach for idempotency. Send a key with the request; if the same key arrives twice, the provider recognises it and does not do the work twice. Stripe has that. Plenty of REST guidance recommends it. So the assumption is that the model provider has it too.
It does not. Anthropic’s Messages API reference and its errors page were both fetched on September 5, 2026 and searched directly for an Idempotency-Key header or any request-level idempotency mechanism. Neither documents one. OpenAI’s current docs for the equivalent endpoint class were searched the same way, with the same result. Both providers are vendors documenting their own APIs, which makes the absence more informative rather than less: this is the page where they would say so.
Where people get burned
What you will find if you search is generic API-design advice saying idempotency keys are a REST best practice, and Stripe’s excellent documentation of its own. Different vendor, different product, different problem. Do not carry the mechanism across by analogy and do not write a lesson, a design doc or a ticket that implies your model provider offers one. Inventing a guarantee is worse than having none, because the code that assumes it will look correct.
This course’s own read on why the absence is unsurprising, offered as reasoning rather than as anything a vendor states: an idempotency key deduplicates a request. Your problem is a partially delivered response. Even a provider that recognised the retry as “the same call” would still have to tell you which four hundred tokens you already have, and that is a resumption protocol, not a deduplication one. The mechanism people wish for would not fix the bug they have.
Which leaves the burden entirely with the application. Nobody is going to recognise the retry for you. You track what was rendered and what was persisted, and then you either resume or you discard and restart. Those are the two moves. Retrying blindly is not a third one.
What automatic retry actually retries
There is retry logic in the provider SDKs, and it is easy to assume it covers this. It does not. Anthropic documents its SDKs retrying “connection errors, rate limits, and 5xx server errors” automatically, “twice by default”. Every one of those is a failure to get a response started. A retried request that has not yet produced output is safe because there is nothing on screen to duplicate.
The transport lesson’s structural fact is what puts you outside that safety. The same page states that “an error can occur after the API returns a 200 response. In that case, error handling doesn’t follow these standard mechanisms”. Once bytes have flowed, the status code is already 200, the automatic retry has already decided this request succeeded, and the recovery is yours. “The SDK retries the call” and “the SDK handles a mid-stream death safely” are unrelated statements.
regenerate() replaces; it does not resume
The AI SDK gives you a named primitive that people reach for here. Its useChat reference documents regenerate as a “function to regenerate the last assistant message or a specific message. If no messageId is provided, regenerates the last assistant message”. Vercel writes and publishes this documentation and sells the platform it leads to, so read the scope of the claim carefully rather than generously.
const { regenerate } = useChat()
// Regenerates the LAST assistant message (or a specific one by id).
// This replaces content; it is not a resume, and the docs do not
// describe it as safe to call while a stream to that message is
// still in flight.
await regenerate({ messageId: brokenMessageId })Two things the documentation does not say, and you should not fill in. It does not describe regenerate as safe to call mid-stream on a connection that is still open, and it does not address what happens to content that arrives from the first stream after the second one has started. Replacement is the documented behaviour. Everything about overlap is yours.
Replacement is also, usefully, the safe option: a retry that replaces a message cannot duplicate it. The duplication bug almost always lives in an append, not in the model.
What a designer does without a guarantee
This section is this course’s own position, not a vendor prescription. No source recommends it; it is what falls out of the absence above.
Invent the run id yourself. Nobody hands you one, so mint one on the client when the turn starts and send it with the request. It is the only thing that makes two streams recognisable as attempts at the same thing, and everything below depends on it.
Make rendering a replacement, keyed by that id. If the UI writes deltas into a slot addressed by run id and attempt number, rather than appending to whatever message node is last, a second attempt cannot concatenate onto the first no matter how badly the timing goes.
// The course's own shape, not an SDK API.
type Attempt = { runId: string; attempt: number; text: string }
// Deltas are written into the slot for (runId, attempt) — never
// appended to "the last message". A late delta from attempt 1
// arriving after attempt 2 has started lands in attempt 1's slot,
// which the UI is no longer showing, instead of in the answer.
function applyDelta(state: Map<string, Attempt>, d: Delta) {
const key = `${d.runId}:${d.attempt}`
const slot = state.get(key) ?? { runId: d.runId, attempt: d.attempt, text: '' }
return new Map(state).set(key, { ...slot, text: slot.text + d.text })
}Pick one policy per surface, and write it down. There are two honest ones. Discard and restart throws away the partial answer, starts a fresh generation, and is safe by construction because there is nothing left to duplicate. It costs a full re-generation and it takes back something the user already read, which on a long answer is its own annoyance. Resume keeps what arrived and asks for the rest, which is cheaper and better, and requires the stream to exist somewhere other than the dead connection — the infrastructure the refresh lesson is entirely about. Choose per surface. Do not let the answer be “whichever the retry button happened to do.”
Reconcile the write path before the retry, not after. If you persisted assistant content as it streamed, the failed attempt’s row is still there and no completion event is coming for it. Close it, mark it failed, or delete it as the first step of the retry. A retry that starts by writing a second row and sorts it out later is how a conversation ends up with one and a half answers in the history and no obvious culprit.
And the side effects. A tool that already ran, ran. The chatbot is advisory, so its award search is a read and re-running it costs a call. The moment a surface like this gains a tool that writes — books, emails, charges — retry policy stops being a UI question, and the answer is not in this course.
Retrieval check
Your retry replaces the message, so it cannot duplicate. Name the duplicate you can still ship.
Check your answer
The one in the database. The screen is a replacement, so the user sees one answer — but the first attempt wrote rows as it streamed, nothing closed them, and the second attempt wrote its own. The conversation history you send with the next turn now contains both, so the model sees itself saying the same thing twice and the user sees an assistant that repeats itself for no visible reason.
The second one is server-side and invisible from the browser entirely: if nothing cancelled the first generation, two generations are running for one answer. Only one has a reader. Both are doing work, and the stop-button lesson established that no vendor documents what the unread one costs.
Check your recall
Answer from memory — no scrolling back.
Hands on
Break the chatbot at token 400 and make the retry survive it
Done when: You can kill a stream after partial output on demand, and the retry that follows produces exactly one answer on screen, exactly one assistant entry in the persisted history, and no second generation running unnoticed.
- Build the failure first. Add a development-only kill switch that destroys the connection after N deltas, with N configurable. Set it to four hundred. Every step below is untestable without this.
- Mint a run id on the client when a turn starts and send it with the request. Log it on both sides. This is the identifier no vendor gives you, so it will not exist unless you create it.
- Change the client so deltas are written into a slot keyed by run id and attempt, not appended to the last message. Then run the kill switch and retry, and confirm the second attempt cannot concatenate onto the first even if a late delta from attempt one arrives.
- Write your policy down in
ARTIFACT.mdin one sentence: discard-and-restart, or resume. If you choose resume, note that you are taking on the infrastructure the refresh lesson describes, and do not tick this step until that is true. - Check the write path. Kill a stream mid-answer, then read your persisted conversation directly. If the failed attempt left an unclosed row, fix the retry to reconcile it before it starts, and verify by killing and retrying three times in a row and counting the rows.
- Count generations, not messages. After one kill and one retry, confirm on the provider side that you are not running two. If you are, the gap is the stop-button lesson’s third piece of wiring, and it is not optional once retry exists.
What this does not cover
The cheaper half of the answer is resumption, and this lesson keeps deferring to it. Making a stream outlive the connection that carried it — a durable store, a stream id the server owns, and a client that reattaches instead of restarting — is the refresh lesson, next. Taking that one step further, so a second viewer and a mid-flight reconnect are the same problem as a refresh, is the two-tabs lesson in the production module.
This lesson also treats every failure as one kind: the stream stopped. It did not, and the four ways it ends badly need four different responses, of which retry is only sometimes the right one. That is the error-taxonomy lesson in the production module. And the case where nothing failed at all — the answer is arriving fine and is simply answering the wrong question — is the partially-wrong-answer lesson that closes this module.
What a retry costs to render, and how to measure the churn a replaced message causes, belongs to Front-end performance under streaming load. This course owns whether the retry is correct. That one owns what it costs to draw.
Read this next — primary source
Messages — Claude API ReferenceAnthropic — vendor documenting its own API. Free. Fetched September 5, 2026, and cited here for what it does not contain
Read this the way this lesson reads it: looking for something. The endpoint that produces every token in your chatbot documents its parameters, its streaming behaviour and its response shape, and documents no request-level idempotency mechanism at all — no Idempotency-Key header, no client-supplied request id that would let a second call be recognised as the same call. Reading the whole reference is how you convince yourself that is true rather than taking this lesson’s word for it, and it is a useful habit: the absence of a mechanism is a design constraint you inherit whether or not you notice it.
Stuck, curious, or think this lesson is wrong? Ask your teaching agent. The lessons are the scaffold; the conversation is where the learning gets unstuck.