frontier

New Frontier Simulator Cuts LLM Inference Latency Error to Under 3% for Disaggregated Serving

Researchers introduce Frontier, a discrete-event simulator for modern LLM inference serving that models disaggregated execution, runtime optimizations, and stateful workloads. On a 16-H800 GPU testbed, Frontier achieves average throughput error below 4% and reduces end-to-end latency error from 44.9% to 6.4% under co-location, and from 51.7% to 2.6% under disaggregation. The simulator scales to over 1K GPUs on commodity CPUs and enables new use cases like SLA-dependent Pareto frontier exploration.