Infrastructure · 6 min

What a reverse proxy actually buffers

Every reverse proxy sits between a client that writes slowly and an origin that would rather not wait. Buffering is the proxy's answer: accept the request or response, hold it in memory or on disk, and forward it at a rate the other side can handle. Most of the time this is invisible. When it is not, it looks like a timeout that nobody can reproduce.

Two directions, two defaults

Request buffering lets the proxy absorb a slow upload before opening a connection to the origin. Response buffering lets it read ahead so a slow client does not tie up an origin worker. Both are reasonable defaults for ordinary web traffic, and both are wrong for long polls, server-sent events, and WebSocket handshakes.

The symptom is distinctive. A 25-second long poll works in curl but returns immediately through the proxy. Or a streaming response arrives in one chunk after ten seconds. The proxy is not broken; it is doing exactly what its default says, which is to accumulate a complete response before forwarding it.

What to check

Look for three settings: response buffering, request buffering, and the read timeout. Turn buffering off for the specific paths that stream, not for the whole site. Raise the read timeout above the longest legitimate poll, and keep a separate idle timeout for upgraded connections. Then verify with a client that prints bytes as they arrive, because a browser will happily hide the difference.

The operational rule

If a protocol depends on timing, it needs a documented proxy profile. Do not rely on a default that was chosen for static assets. Write the profile down, test it through the same path users take, and treat any new hop as a change to the timing contract.

← Back to the notes