Tiered inference distributes large‑language‑model computation across a hierarchy of devices—from on‑device to edge to datacenter—to reduce cost, latency, and improve privacy. Most existing systems freeze model checkpoints and use a confidence signal emitted by a forward‑deployed model to decide whether to resolve a query locally or to escalate it. This paper derives a closed‑form feasibility condition that tells when no confidence threshold can make a tiered cascade better than sending every query to the backstop model. We model the routing decision as a binary test and separate local from escalation cost with an architecture cost ratio $\kappa$, yielding a feasibility bracket: below a signal‑free floor, any cascade is feasible regardless of signal quality; above a perfect‑signal ceiling, no cascade can be feasible. Reading along the $\kappa$ axis the condition inverts: when local compute is billed, cost becomes the binding factor, matching recent empirical findings; when local compute cost is sunk (decentralized hardware), feasibility is unconditional and signal quality determines the achievable gain. We translate the condition into three design requirements; existing trained primitives satisfy only one of them in isolation. We propose, as future work, a co‑designed architecture and a protocol to evaluate it. Importantly, the condition and its inversion are independent of any specific architecture and constitute the main contribution of the paper.
Review: The work offers a concise theoretical criterion for assessing the viability of tiered LLM inference, giving system designers a clear guideline and highlighting the interplay between signal quality and cost under different cost models.