How Nginx's Default gzip Buffering Turned Our Claude Streaming Responses Into a 28-Second Freeze

10:47 AM. A support ticket comes in titled "AI generator broken, just spins forever." Then another. By 11:10 there are nine of them, all describing the same thing: the draft generator hangs with no text, then the entire response appears at once, line breaks and all, like someone pasted it in. Nobody touched the model. Nobody touched the prompt. The only thing that shipped this morning was an infra change nobody thought was risky.

the scramble

The feature is a drafting assistant inside the app: a user picks a template, Claude streams a draft back token by token over server-sent events, and the UI renders it live so it doesn't feel like a 20-second wait staring at a spinner. It had worked this way for four months. The only change that morning was moving the endpoint behind the company's shared Nginx reverse proxy, which already fronted every other internal API for TLS termination and centralized access logs. Previously this one service sat directly behind the load balancer.

First theory in #eng-oncall: Claude's API is degraded. It's the obvious suspect when something that calls an LLM suddenly feels slow. Someone checks status.anthropic.com, clean. Someone else pulls the Datadog dashboard for request latency to the Messages API, nothing unusual, p95 time-to-first-token sitting right where it's been all week. A direct curl against the Anthropic API from a laptop streams back tokens in well under a second and keeps going smoothly. Claude is fine.

Second theory: the new proxy is buffering the upstream response, a classic Nginx gotcha. Someone greps the shared config and finds proxy_buffering off; already set on this location block, added two incidents ago for a different streaming endpoint and left in place as a known good default. It should already prevent exactly this. It doesn't fix anything. The draft generator still freezes, then dumps.

the hunt

With the two obvious theories dead, someone stops guessing and starts diffing. Hit the endpoint directly against the Node service, bypassing Nginx entirely, and the stream behaves exactly like it always has. Hit it through the proxy and it stalls. Same request, same backend, same model call. The only variable left is whatever sits in that one hop.

# direct to the Node service
curl -N -i https://draft-svc.internal/generate -d '{"templateId":"follow-up"}' | head -10
HTTP/1.1 200 OK
content-type: text/event-stream
cache-control: no-cache
connection: keep-alive

# through the shared Nginx proxy
curl -N -i https://api.example.com/draft/generate -d '{"templateId":"follow-up"}' | head -10
HTTP/1.1 200 OK
content-type: text/event-stream
content-encoding: gzip
cache-control: no-cache
connection: keep-alive

content-encoding: gzip on a server-sent events stream is the kind of header nobody reads twice. The shared proxy's config turns on gzip compression for every response above a minimum size, to shrink the large JSON payloads most of the other internal APIs return. Nobody had excluded this content type from that rule when the draft endpoint was added to the same location block, because on paper gzip just makes responses smaller. Nobody wrote "disable compression" anywhere on the migration checklist, because nobody thought of SSE as a response that gets compressed at all.

The mechanics: Nginx's gzip filter module operates on a chain of fixed-size internal buffers, gzip_buffers 32 4k by default, meaning each buffer in the chain holds 4KB. The module won't hand a compressed chunk downstream to the client until a buffer fills or the upstream connection closes. A typical draft streams out at roughly 20 to 40 tokens a second, around 100 to 200 bytes a second of raw text. Filling a single 4KB buffer at that rate takes somewhere between 20 and 40 seconds, which lines up almost exactly with the freeze times in the support tickets. proxy_buffering off controls whether Nginx buffers the full response before forwarding it from the upstream, a completely different layer. It does nothing to stop the gzip filter from buffering its own output after that.

the find

The draft endpoint's response was getting compressed by a module that has no concept of streaming, and the gzip filter's internal buffering, not proxy buffering, was holding every token until it had enough bytes to flush a chunk. For a request that completes in a second, this is invisible. For one that trickles out over 20 to 30 seconds, it turns a live stream into a single chunk delivered right as the upstream connection to Claude closes, because that's the only other event that forces a flush.

the fix

The actual fix is one line: exclude text/event-stream from the location block's gzip types, so Nginx never puts this response through the compression filter in the first place.

location /draft/generate {
    proxy_pass http://draft-svc.internal;
    proxy_buffering off;
    proxy_http_version 1.1;
    proxy_set_header Connection "";

    # gzip was inherited from the parent server block's gzip_types list,
    # which was written for JSON APIs and never excluded SSE responses
    gzip off;
}

Scoping gzip off to just this location block was the simplest fix. There was no upside to compressing a response that's already small per chunk and latency sensitive by design. As a second layer of defense, in case the next team moves this endpoint behind a different proxy and forgets this whole story, the Node service now also sends X-Accel-Buffering: no on the response, an Nginx-specific header that disables response buffering for that request regardless of the location block's own settings.

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

app.post("/draft/generate", async (req, res) => {
  res.setHeader("Content-Type", "text/event-stream");
  res.setHeader("Cache-Control", "no-cache");
  res.setHeader("Connection", "keep-alive");
  res.setHeader("X-Accel-Buffering", "no"); // belt and suspenders against any future proxy

  const stream = client.messages.stream({
    model: "claude-opus-5",
    max_tokens: 2048,
    messages: [{ role: "user", content: buildPrompt(req.body.templateId) }],
  });

  for await (const event of stream) {
    if (event.type === "content_block_delta" && event.delta.type === "text_delta") {
      res.write(`data: ${JSON.stringify({ text: event.delta.text })}

`);
    }
  }

  res.write("data: [DONE]\n\n");
  res.end();
});

The header is advisory, Nginx respects it, a misconfigured proxy that ignores it would still break streaming, so it's a safety net rather than the actual control. The actual control is not letting a shared compression rule apply to a content type it was never written for.

the aftermath

Nothing about the Node service, the Anthropic SDK, or the model call was ever broken. The proxy did exactly what its gzip module is designed to do, compress a response before sending it, on a response where that single design decision was the entire bug.