cd ..

$ cat ~/field-notes/api-timeouts-deadlines.md

API Timeouts: Deadlines, Cancellation, and Why Defaults Fail

5 min read 0x.run

An outbound HTTP call without a timeout is a resource leak waiting for a network failure. A timeout added at random is only slightly better.

Production systems need a budget: how long the entire operation may take, how that time is divided, and what gets canceled when the budget expires.

"The Timeout" Is Several Timeouts

An HTTP request has phases:

DNS lookup
TCP connection
TLS handshake
request body write
waiting for response headers
response body read

Different failures stall in different phases. A server can accept a TCP connection immediately and then never send headers. Or send headers and drip one response byte every 20 seconds.

Useful clients expose separate controls:

connect timeout         bound DNS/TCP/TLS setup
response-header timeout bound time to first response headers
read/write timeout      bound stalled socket I/O
overall deadline        bound the complete operation

An overall deadline is the final safety net. Phase limits make errors more specific and prevent slow trickles from resetting one simplistic timer forever.

Timeout vs Deadline

A timeout is a duration: "allow 800 ms."

A deadline is a point in time: "finish before 14:03:17.450."

Deadlines compose better across services. If an API begins with a 2-second budget and spends 650 ms on authentication and a database query, the next service does not still have 2 seconds. It has at most 1.35 seconds—and should receive less so the caller has time to handle its response.

client budget:                  2000 ms
edge + authentication used:     200 ms
service A database used:         450 ms
reserve for response handling:   150 ms
maximum downstream budget:      1200 ms

Passing a fresh timeout at every hop creates latency amplification. A chain of four services can keep working long after the original client has left.

Cancellation in JavaScript

Modern fetch accepts an AbortSignal:

async function fetchProfile(userId, parentSignal) {
    const timeout = AbortSignal.timeout(800);
    const signal = AbortSignal.any([parentSignal, timeout]);

    const response = await fetch(
        `https://profiles.internal/users/${encodeURIComponent(userId)}`,
        { signal },
    );

    if (!response.ok) {
        throw new Error(`profile service returned ${response.status}`);
    }

    return response.json();
}

Combining signals means either the local 800 ms limit or the caller's cancellation can stop the request.

Always classify cancellation separately from service failures:

try {
    return await fetchProfile(id, request.signal);
} catch (err) {
    if (err.name === 'AbortError' || err.name === 'TimeoutError') {
        metrics.increment('profile_request_timeout');
        throw new ServiceUnavailableError('profile lookup timed out');
    }
    throw err;
}

The exact error type varies by runtime and by which signal aborted. Test it in the version you deploy.

Cancellation in Go

Go's context makes request-scoped deadlines explicit:

func loadProfile(ctx context.Context, userID string) (*Profile, error) {
    ctx, cancel := context.WithTimeout(ctx, 800*time.Millisecond)
    defer cancel()

    url := "https://profiles.internal/users/" + url.PathEscape(userID)
    req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
    if err != nil {
        return nil, err
    }

    resp, err := httpClient.Do(req)
    if err != nil {
        return nil, fmt.Errorf("profile request: %w", err)
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        return nil, fmt.Errorf("profile status: %s", resp.Status)
    }

    var profile Profile
    return &profile, json.NewDecoder(resp.Body).Decode(&profile)
}

Pass the incoming request context down. Do not replace it with context.Background() in database and HTTP helpers—that disconnects the work from the request lifecycle.

Configure transport phase limits too:

var httpClient = &http.Client{
    Timeout: 2 * time.Second, // whole exchange, including body read
    Transport: &http.Transport{
        DialContext: (&net.Dialer{
            Timeout:   300 * time.Millisecond,
            KeepAlive: 30 * time.Second,
        }).DialContext,
        TLSHandshakeTimeout:   300 * time.Millisecond,
        ResponseHeaderTimeout: 800 * time.Millisecond,
        IdleConnTimeout:       90 * time.Second,
    },
}

Reuse the client and transport. Creating one per request throws away connection pooling.

Stop Work When the Client Leaves

If a browser disconnects, the reverse proxy and application should cancel downstream work. Otherwise a timed-out request may continue running expensive queries, generating a report no one will read.

Cancellation is cooperative. Passing a signal does nothing if a library ignores it. Confirm that your database driver, HTTP client, and queue producer accept the request context and actually interrupt operations.

Some database engines cancel the client wait but need an explicit protocol message to stop the query on the server. Observe both application latency and database activity.

Retries Must Fit the Same Budget

This is broken:

overall API budget: 2 seconds
attempt timeout:    2 seconds
maximum attempts:   3

The worst case is over 6 seconds after backoff and overhead.

Instead, calculate from the remaining deadline:

remaining: 1200 ms
attempt 1:  500 ms
backoff:     50 ms
attempt 2:  500 ms
reserve:    150 ms

Only retry failures likely to be transient, and only retry operations that are idempotent or protected by an idempotency key. When too little budget remains for a useful attempt, fail immediately.

Choose Timeouts From Data

Timeouts should reflect a service objective and observed latency distribution, not a copied constant.

If a dependency normally responds at p99 in 180 ms, a 10-second timeout hides a serious incident. A 190 ms timeout may cause unnecessary failures during normal variation. Start with a value that leaves room inside the caller's deadline, then adjust using:

dependency latency p50/p95/p99
timeout count by operation and phase
remaining budget at timeout
connection pool wait time
requests canceled by caller
work completed after caller cancellation

Do not use averages. Averages hide the slow tail that triggers timeouts.

Align Every Layer

A common stack has several clocks:

browser → CDN → load balancer → app → database/API

Inner operations should expire before outer ones, leaving time to return a useful error.

database statement:   700 ms
downstream service:   900 ms
application request: 1500 ms
load balancer:       1800 ms
client:              2200 ms

These values are illustrative, not universal. The important property is ordering and a deliberate reserve.

The Bottom Line

Timeouts define how long failed work is allowed to consume resources.

The rules:

  • Set explicit phase limits and an overall deadline
  • Propagate the caller's cancellation through every dependency
  • Give downstream calls only the remaining budget, minus a reserve
  • Reuse HTTP clients so timeouts do not destroy connection pooling
  • Make retries fit inside the original deadline
  • Stop queries and other work when the client disconnects
  • Choose values from tail latency and service objectives
  • Make inner layers expire before outer layers

A timeout that merely returns an error is incomplete. The real goal is to stop the work, free the resource, and leave enough time for the system to recover cleanly.