NeFut Logo NeFut
Admin Login

[CS.AI] Trie Automata for Constrained Decoding over Large Finite Sets

Published at: 2026-08-15 22:00 Last updated: 2026-08-16 07:03
#Trie #Automata #Constrained Decoding #Large Finite Sets #Aho-Corasick

Large language models increasingly need to generate structured outputs that conform to predefined schemas, with one common constraint being selection from a finite set of valid strings. Current constrained decoding systems handle this through general-purpose grammar compilation, which becomes prohibitively slow as the number of valid values grows into the thousands. We introduce the trie automaton, a specialized mechanism that exploits finite-set structure (shared prefixes, bounded depth, known cardinality) via Aho-Corasick multi-pattern matching to precompute per-node token masks. The trie achieves 7X faster per-step valid-token computation (0.65 us vs. 5.8 us) compared to XGrammar, one of the primary backends in vLLM and SGLang, and 2--6.5X faster compilation at K = 300. Because precomputed masks enable a stateless serving path that bypasses the guided decoding pipeline, this advantage compounds in batch serving: end-to-end vLLM throughput reaches 219 req/s vs. XGrammar's 7.5 req/s at batch size 256 (29X). Across seven tokenizer families (32K--262K vocabulary), the trie maintains sub-100ms compilation up to K = 10,000 and flat per-step cost regardless of set size, while guaranteeing 100% output validity. Blogger's Review: The trie automaton brings an efficient solution for constrained decoding in large language models, especially when dealing with large finite sets. Its precomputed mask mechanism makes the decoding process faster and more efficient, with a wide range of applications.

Original Source: https://arxiv.org/abs/2608.12574

[h] Back to Home