LLM serving optimization typically benchmarks many configurations and relies on heavy profilers when latency targets are missed. We advocate for the reverse discipline: estimation serves as the analytical foundation of profiling—without it, optimization degenerates into grid search. Floor First is a residual-driven triage workflow. Each decode step is modeled as a five-dimensional resource vector (HBM bytes, FLOPs, network bytes, network messages, KV capacity); summing within a resource and maximizing across resources yields an optimistic floor, while the plain sum provides a pessimistic one. The measurement's position within this [max, sum] interval indicates overlap quality before any profiler is opened, and profilers escalate only on residuals exceeding a stated threshold. Deployment alternatives are compared by wall ordering—determining which resource wall binds first as load grows—rather than by point benchmarks. The approach is compositional: new attention or state-space variants enter by declaring one module, and the workflow ships as a zero-dependency calculator along with an agent skill that enforces discipline in agentic optimization loops. As a case study, we analyze a DeepSeek-V3.2-style 671B MoE/MLA model on 16 NVIDIA H20 GPUs, whose ridge point of ~74 FLOP/byte (vs ~590 for H100) makes it an extreme decode-oriented part. The floors show TP16 decoding is KV-capacity-limited to ~70 concurrent 8K requests; sparse attention eliminates the KV-bandwidth term but not the capacity wall; an EP16+DP-attention layout accepts slightly worse same-batch weight traffic for an order-of-magnitude higher capacity wall (~644)—while single-stream latency favors TP by 2.4x. Thus, the layout judgment is a computable function of the operating point, explaining why production deployments on identical hardware have shipped opposite attention layouts.
Blogger's Review: This article introduces an innovative Floor-First triage method to optimize LLM service performance. By modeling resources in five dimensions, the authors demonstrate how to make effective deployment decisions using estimation and wall ordering without relying on heavy profiling tools. This approach not only enhances efficiency but also leads to better performance tuning in practical deployments, making it a valuable discussion point for researchers and engineers alike.