How soon before hitting the network are request hooks processed?

[austind]

I'm building an async client based on httpx for a service that uses request signing. Signatures must fall within a specific timestamp or risk 401 response. This is not a problem for low volume requests, but high volume requests run the risk of failing authentication due to sufficient clock skew (i.e., the request was signed >5min before it hit the network, as it was sitting in the connection pool waiting to be processed).

I thought a way around this might be using event hooks, and at first they seemed to work well. Though in at least one high volume test, I ran into a bunch of 401s. The docs aren't clear exactly when the request hook is processed, so I'm not sure what to make of it.

Are request hooks a good way to handle this? Any other thoughts appreciated!

[ryux1]

A request hook is the right public extension point for signing, but it runs a little earlier than “socket write.” HTTPX invokes it immediately before _send_single_request, and that then delegates to the transport; connection-pool acquisition happens inside the transport. So time spent waiting for a pool slot is after your hook and can age the signature. (client path, transport handoff)

A practical pattern is:

async def sign(request: httpx.Request) -> None:
    # Recompute both values from the final request bytes on every attempt.
    request.headers["X-Timestamp"] = current_timestamp()
    request.headers["Authorization"] = make_signature(request)

timeout = httpx.Timeout(30.0, pool=5.0)
limits = httpx.Limits(max_connections=50)
client = httpx.AsyncClient(
    timeout=timeout,
    limits=limits,
    event_hooks={"request": [sign]},
)

The short pool timeout is important: it bounds how stale a hook-generated signature can become while queued. On httpx.PoolTimeout, retry at the application layer so a fresh request attempt is signed again. Also put an asyncio.Semaphore around your application’s calls, sized to the client’s intended concurrency, to apply backpressure before requests enter HTTPX. HTTPX documents pool timeout specifically as the wait to acquire a connection: https://www.python-httpx.org/advanced/timeouts/#fine-tuning-the-configuration

Allow margin for connection/TLS setup and request-body writes too. If the protocol truly requires signing only after a connection has been acquired, HTTPX currently has no public event hook at that exact point; that would require a lower-level transport/httpcore integration. For a five-minute validity window, bounded pool waiting plus application backpressure should normally avoid that complexity.

[ryux1]

A request hook runs too early to guarantee a fresh signing timestamp at high concurrency.

In both the synchronous and asynchronous client paths, HTTPX invokes the request hooks in _send_handling_redirects() and only afterwards calls _send_single_request(). _send_single_request() then enters the transport, and the default transport/httpcore layer is where connection-pool acquisition happens. A request can therefore sit waiting for an available pooled connection after your hook has signed it. The hook also runs once per redirect request, but that does not move it past pool acquisition.

A custom Auth flow has the same fundamental timing issue: it prepares/yields the request before the transport sends it. Wrapping HTTPTransport externally does not necessarily solve it either, because the wrapper still runs before the wrapped transport waits for its pool slot.

The robust options are generally to:

• bound application concurrency so requests do not queue longer than the server’s signature window;
• set a suitably short pool timeout and, after PoolTimeout, rebuild/re-sign a fresh request before retrying;
• use a service-specific transport/httpcore customization that signs only after acquiring a connection, if the service volume truly requires that precision; or
• if you control the protocol, sign stable request data plus a nonce and use a server-side freshness window that tolerates normal client queuing.

Be careful with automatic retries: retrying the same already-signed Request preserves the stale timestamp, so the retry path must explicitly generate a new signature (and usually a new nonce). For your existing design, limiting concurrency plus a pool timeout comfortably below five minutes is probably the smallest reliable change.

[austind]

@ryux1 Thank you for the very thorough answer. Much appreciated!