Surviving the infrastructure
The transport lesson picked how bytes leave your server; this is about them arriving — every layer in between defaults to holding a response until it is complete, from nginx to Azure Application Gateway to a CDN inspecting a prefix, and each one is doing a job somebody else is paying for.
Your stream works. You defeated a local nginx in the transport lesson, you watched the first byte arrive early, and you have the header list that makes it happen. Now the chatbot goes into a portfolio company’s product, and between your process and the browser there are four things you did not configure, cannot see, and will not be given access to.
Every one of them defaults to holding your response until it is complete. Not because anyone was careless. Because holding a response is what those products are for.
Everybody buffers, and everybody has a reason
Start with the one you already met. nginx documents proxy_buffering as defaulting to on, and the escape hatch as a response header: “Buffering can also be enabled or disabled by passing ‘yes’ or ‘no’ in the ‘X-Accel-Buffering’ response header field. This capability can be disabled using the proxy_ignore_headers directive.” In the transport lesson that was a column in a decision table. Here it is the obstacle, and the last clause is the whole difference: your fix is a request the operator has already been given the tools to ignore.
Azure Application Gateway is the clearest published statement of why a layer would do this on purpose. Microsoft, documenting its own product:
“By default, the Response buffering is enabled on Application Gateway which is useful to accommodate slow clients.”
The mechanism is spelled out on the same page: the response buffer “can collect all or parts of the response packets sent by the backend server, before delivering them to the clients,” so that backend connections “can be closed once Application Gateway receives complete response”. Read that as an engineer rather than as a victim. The gateway is protecting your origin from a slow client holding a connection open for a minute. That is a real problem, and on a normal API it is a good trade. Your ninety-second stream is the case where the trade inverts.
Three operational details from that page matter more than the default itself, because they determine whether the fix is even available to the person you are talking to. The settings are resource-level and, in Microsoft’s words, “can’t be managed separately for each listener” — so turning buffering off for your chatbot turns it off for everything else behind that gateway. The page states that “currently, these changes aren’t supported through the Azure portal,” leaving CLI, PowerShell or ARM, which is a different approval path in most organisations. And request buffering “can’t be disabled if you’re running the WAF SKU,” though the page is explicit that “Response buffering isn’t impacted by the WAF.” The knobs are enableResponseBuffering and enableRequestBuffering, under globalConfiguration.
The CDN layer says the same thing in security language. Cloudflare — also a vendor documenting its own product — ships Response Body Buffering with a default of “Standard,” which “allows Cloudflare products to inspect a prefix of the response body,” against “None: strictly no buffering”, and its own changelog warns that None “may break security functionality that requires body inspection, including the Web Application Firewall (WAF) and Bot Management”. You are not asking a security team for a checkbox. You are asking them to stop inspecting response bodies.
Not every layer is hostile, and knowing which are not is half of being useful in the room. AWS CloudFront, documenting itself, states that it “supports only the chunked value of the Transfer-Encoding header” and returns such a response “to the client as the object is received at the edge location”. That is a CDN that passes a stream through by design.
The timeout is between bytes, not across the response
The second failure looks nothing like the first. The stream starts correctly, runs for a while, and dies mid-answer with no error — typically while a long tool call is running and nothing is being written.
nginx again: proxy_read_timeout defaults to 60 seconds, and “the timeout is set only between two successive read operations, not for the transmission of the whole response. If the proxied server does not transmit anything within this time, the connection is closed.” CloudFront measures its origin response timeout the same way, as the gap between packets rather than the length of the response.
That framing is good news and it is the actionable half of this lesson. A stream may run for an hour if it writes something often enough. So write something often enough. An SSE comment line — a line beginning with a colon — is ignored by every conforming parser and resets every between-bytes timer between you and the browser.
The transport itself has a timer too, and it is one layer below anything you can configure in a proxy. RFC 9114 (HTTP/3) states that “if the QUIC connection remains idle (no packets received) for longer than this duration, the peer will assume that the connection has been closed”, with the duration declared by each endpoint during the QUIC handshake. Same lesson, different layer: silence is what kills a long response, and a keep-alive is not a nicety.
Which HTTP you are speaking changes the failure
A deploy guide that tells you to set Transfer-Encoding: chunked is giving you HTTP/1.1 advice, and it is worth knowing exactly how wrong that gets as you move up. MDN warns that “HTTP/2 disallows all uses of the Transfer-Encoding header… Usage of the header in HTTP/2 may likely result in a specific protocol error”. HTTP/3 is blunter still: “Transfer codings… are not defined for HTTP/3; the Transfer-Encoding header field MUST NOT be used”. Streaming on those versions happens in protocol frames. The word “chunked” is an HTTP/1.1 detail, not a synonym for streaming.
The version can also be downgraded underneath you without anyone touching your code. nginx’s proxy_http_version defaults to 1.1 since 1.29.7, and defaulted to 1.0 before that. On an older install, the hop between the proxy and your app is HTTP/1.0, which has no chunked framing at all — so the proxy has no way to forward your response incrementally even with buffering disabled. Check the nginx version, not just the config.
Connection limits are the last version-dependent trap, and this is a place where two good sources disagree in a way worth carrying. MDN says the limit is six per browser and domain outside HTTP/2, and on HTTP/2 is “negotiated between the server and the client (defaults to 100)”. The specification says something narrower: RFC 9113 says of SETTINGS_MAX_CONCURRENT_STREAMS that “initially, there is no limit to this value,” and only that “it is recommended that this value be no smaller than 100, so as to not unnecessarily limit parallelism”, while RFC 9114 says “at least 100 request streams SHOULD be permitted at a time”. A recommendation is not a default. The practical reading: the number is negotiated by two parties, neither of which is you, so design for the six-connection floor and never write code that assumes a hundred.
Platform limits are not general truths
Everything above is either a specification or a vendor documenting a product you can read the docs for today. Platform limits are a different category and they are where this module refuses to guess.
Whether AWS Application Load Balancer buffers responses, and whether its idle timeout is measured between bytes or across the connection, was not checked in this course’s research passes. For AWS API Gateway, the quotas page was fetched and the returned tables did not contain the integration-timeout row, so the figure everyone repeats for it is unverified here and does not appear anywhere in this course. Lambda response-streaming limits, Google Cloud Run streaming behaviour, and function-duration caps on Vercel, Netlify and Cloudflare Workers are all unverified for the same reason.
Treat that as the method, not as a gap to be embarrassed about. A platform limit is a fact about one platform on one date. Read the vendor’s own current page, write the number down with the date you read it, and re-check it before you quote it in a design review. The specs in this lesson have been stable for years; the platform numbers move, and a confidently recited timeout that changed last quarter is worse than admitting you have to look.
One repeat warning, because this module reuses the nginx source and the claim is magnetic: do not tell anyone that nginx documents gzip as breaking SSE. Its gzip module page says nothing about streaming or flushing. The transport lesson works through why, and what to cite instead.
Check your recall
Answer from memory — no scrolling back.
Retrieval check
Your response sets X-Accel-Buffering: no and the stream still arrives in one lump, with a 200 and no errors. Name three distinct explanations, all consistent with what you observed.
Check your answer
First, the header was honoured and something else is buffering. Streaming responses commonly pass through more than one intermediary, and the nginx header means nothing to an application gateway or a CDN — each layer has its own switch and its own default.
Second, the header was deliberately discarded. proxy_ignore_headers exists precisely so an operator can stop upstream applications from steering their proxy, and it is a reasonable thing for them to have set.
Third, the hop cannot stream at all. If nginx is proxying to your app over HTTP/1.0 — the default before 1.29.7 — there is no chunked framing on that hop, so there is nothing for buffering to be turned off from.
Three causes, one symptom, and none of them visible in your application logs. That is why the next step is always to measure at each hop rather than to change a header and hope.
Hands on
Map every layer, then produce the failure on purpose
Done when: ARTIFACT.md carries an infrastructure map with one row per layer between your app and the browser — the layer, its buffering default, its between-bytes timeout, where you read that, and the date — plus a recorded observation of the same endpoint measured at two different hops.
- List every hop between your process and the browser, in order. Platform runtime, load balancer or gateway, CDN, and anything a portfolio company would add. If you do not know, say so in the row rather than leaving it blank — “unknown, nobody asked” is a finding you can act on and an empty cell is not.
- For each hop, find the vendor’s own current page for its buffering behaviour and its timeout, and record the quote, the URL and today’s date. If the page does not state it, write “not documented” rather than the number you remember reading somewhere. That row is the honest one.
- Measure the same streaming endpoint at two hops: directly against your app, and through the full chain, with
curl -Nand timestamps on the first and last byte. The difference between those two numbers is the only measurement in this task, and it is the one that settles arguments. - Add a keep-alive to your stream: an SSE comment line on an interval comfortably under the shortest between-bytes timeout in your map. Then prove it works by making the model go quiet on purpose — stall a tool call — and confirming the connection survives. Remove the keep-alive and confirm it dies. Both directions, or the test proved nothing.
- Write the one-paragraph ask you would send to a team that owns a layer you cannot change. It has to name the layer, quote its own documentation back to it, state what the default is protecting, and propose the narrowest change that works. Bring that paragraph into the chat and I will argue it from the platform team’s side.
What this does not cover
This lesson is about bytes arriving. It says nothing about what it costs to render them once they do — re-parse cost per token, memoization boundaries, INP measurement, performance budgets, or anything you would put on a dashboard. That work belongs in full to Front-end performance under streaming load, which sits next to the measurement discipline that justifies it. The one number in the task above exists to compare two hops of the same request, not to establish a budget.
It also stops at a single connection carrying a single response. The moment a run outlives the connection that started it — because the user refreshed, because a proxy cut it, because they opened the chat in a second tab — the model in your head has to change, and that is the two-tabs-one-run lesson next. The stop-button and refresh-survival lessons in the control module cover the interaction side of the same event.
Read this next — primary source
Configure Request and Response Buffers — Azure Application GatewayMicrosoft — vendor documenting its own product. Verified 2026-09-03, spot-checked 2026-09-05
This lesson takes its central quote from it. Read the whole page because it is the clearest example anywhere of a load balancer that waits for the complete response deliberately, states the benefit it is buying, and then tells you the operational cost of turning that off: the setting is resource-level rather than per-listener, it is not exposed in the portal at all, and one half of it cannot be disabled on the WAF SKU. That combination — a documented default, a stated reason, and a change you cannot make from the console — is the shape of every infrastructure conversation in this module.
Stuck, curious, or think this lesson is wrong? Ask your teaching agent. The lessons are the scaffold; the conversation is where the learning gets unstuck.