#latency

A debugging story showing how modeling replayed traffic as a Poisson process, rather than fixed intervals, explained a 2x p99 latency gap in GPU model serving.

Notes on CPU and disk latency, and on OS time slicing and CPU scheduling.

Explains the core problems of distributed systems: partial failures, unreliable networks and clocks, synchronous versus asynchronous networks, and long-tail latency.

Explains how hedged requests reduce long-tail latency in distributed systems by sending a duplicate request to another replica after a p95 delay, with the trade-offs around idempotency and choosing the delay.