cd ..

$ cat ~/field-notes/http-compression-gzip-brotli.md

HTTP Compression: gzip, Brotli, and the Bytes You Shouldn't Compress

5 min read 0x.run

The fastest byte is the one you never send. Minification removes unnecessary source characters; HTTP compression finds repeated byte patterns and encodes them more efficiently. You usually want both.

The trap is turning compression on globally and assuming the job is finished.

How Negotiation Works

The browser tells the server which encodings it understands:

GET /app.js HTTP/1.1
Host: example.com
Accept-Encoding: gzip, deflate, br

The server chooses one and labels the response:

HTTP/1.1 200 OK
Content-Type: application/javascript
Content-Encoding: br
Vary: Accept-Encoding

Content-Encoding describes the representation on the wire. The browser decompresses it before JavaScript sees the body.

Vary: Accept-Encoding tells shared caches that compressed and uncompressed responses are different variants. Without it, a cache can serve Brotli bytes to a client that did not request Brotli.

gzip vs Brotli

Both are general-purpose compression formats used by browsers.

gzip:
  + fast to encode
  + universal HTTP support
  + good default for dynamic responses

Brotli (br):
  + usually produces smaller text assets
  + high levels are expensive to encode
  + ideal when static files can be compressed during the build

For dynamic JSON, gzip at a moderate level is often a good latency/CPU trade-off. For versioned JavaScript and CSS, generate .br and .gz files once during the build and serve the best supported variant.

Do not run maximum Brotli compression on every live request. Saving another few percent is rarely worth tying up a CPU core.

What to Compress

These formats usually compress well:

text/html
text/css
text/plain
application/javascript
application/json
application/xml
image/svg+xml

These are already compressed and usually do not:

image/jpeg
image/png
image/webp
image/avif
video/mp4
audio/mpeg
application/zip
application/gzip

Compressing a JPEG again consumes CPU and can make it slightly larger. Compression decisions should use the response MIME type, not the filename.

Skip Tiny Responses

Compression adds headers, wrapper bytes, memory allocation, and CPU work. A 70-byte 204-style response will not become meaningfully faster.

A threshold around 1 KB is a reasonable starting point for dynamic responses. Measure your own traffic: latency, network cost, and server CPU determine the right threshold.

import compression from 'compression';
import express from 'express';

const app = express();

app.use(compression({
    threshold: 1024,
    level: 6,
    filter(req, res) {
        // Allow a route to opt out for security or CPU reasons.
        if (res.getHeader('X-No-Compression')) return false;
        return compression.filter(req, res);
    },
}));

Middleware order matters. Compression needs to see the final response headers and body stream.

Configure nginx for Dynamic Text

gzip on;
gzip_comp_level 5;
gzip_min_length 1024;
gzip_vary on;
gzip_proxied any;

gzip_types
    text/plain
    text/css
    application/json
    application/javascript
    application/xml
    image/svg+xml;

Do not add image/png or archives to gzip_types. Also confirm whether your CDN already compresses at the edge; double-compression is a configuration bug, not an optimization.

Precompress Static Assets

Hashed assets do not change after deployment, so compress them once:

# Keep the original files and add encoded variants.
gzip -9 -k dist/app.8f3c1.js
brotli --quality=11 --keep dist/app.8f3c1.js

# Produces:
# app.8f3c1.js
# app.8f3c1.js.gz
# app.8f3c1.js.br

The web server or CDN can select .br, then .gz, then the original according to Accept-Encoding. Precompression moves expensive work out of the request path and makes response time predictable.

Use long cache lifetimes only for content-addressed filenames:

Cache-Control: public, max-age=31536000, immutable

Never use a year-long immutable cache for /app.js if deploying new code overwrites the same URL.

Streaming Changes the Trade-off

Compressors buffer enough input to find patterns. That can delay the first bytes of a streamed response.

For server-sent events, frequent flushing is more important than a smaller stream. Disable compression or explicitly flush after each event if your stack supports it. For large generated reports, streaming compression is useful because it reduces bytes without buffering the entire document in memory.

Test time to first byte, not just final file size.

Compression Can Leak Secrets

Compression ratio reveals when attacker-controlled text matches secret text in the same response. Repeated guesses can turn response sizes into an oracle. This is the family of attacks commonly associated with BREACH.

The practical defense is architectural:

  • Do not reflect attacker input next to CSRF tokens or other secrets
  • Do not put secrets in URLs
  • Rotate CSRF tokens and mask them where your framework supports it
  • Disable compression on responses that combine secrets with reflected input
  • Keep cookies out of response bodies

TLS encryption does not hide the length of the compressed response.

Verify What Is Actually Served

# Ask for Brotli or gzip and inspect headers.
curl -I --compressed https://example.com/app.js

# Compare transferred sizes.
curl -sS -o /dev/null -w '%{size_download}\n' \
  -H 'Accept-Encoding: identity' https://example.com/app.js

curl -sS --compressed -o /dev/null -w '%{size_download}\n' \
  https://example.com/app.js

Check HTML, API responses, and large assets separately. A CDN dashboard saying "compression enabled" does not prove every cache key and content type is correct.

The Bottom Line

Compression is a negotiation and caching feature, not a single boolean.

The rules:

  • Compress text; skip images, video, and archives that are already compressed
  • Skip tiny bodies
  • Precompress immutable static assets with Brotli and gzip
  • Use moderate levels for dynamic responses
  • Always send Vary: Accept-Encoding when representations differ
  • Measure time to first byte for streams
  • Separate secrets from attacker-controlled response content
  • Verify the bytes and headers through the CDN, not just locally

Configured well, HTTP compression is one of the cheapest performance wins available. Configured blindly, it burns CPU, confuses caches, and occasionally creates a security problem.