#queuing-theory

Queuing Theory In Model Serving
A debugging story showing how modeling replayed traffic as a Poisson process, rather than fixed intervals, explained a 2x p99 latency gap in GPU model serving.