We introduce Mahalanobis‑Ensemble Decoding (ME‑Decoding), a new framework that casts Large Language Model token selection as ensemble pruning. Existing strategies rely on scalar probabilities, overlooking geometric semantic relations and causing redundant generation paths. Geometry‑aware methods that do exist often require heavy optimization or direct reweighting of original probabilities, leading to large computational costs and inference instability. ME‑Decoding formulates decoding as a subset‑optimization problem with a Mahalanobis‑distance‑driven objective that promotes semantic diversity while preserving high‑probability tokens. A token similarity matrix is built using an adaptive‑bandwidth kernel over token embeddings, enabling dynamic discounting of redundant paths. An efficient greedy selection algorithm with near‑linear complexity under early stopping is proposed, accompanied by theoretical approximation guarantees. The approach is a plug‑and‑play module with negligible inference overhead. Extensive experiments on reasoning and generation tasks consistently show strong performance improvements.
Review: ME‑Decoding leverages Mahalanobis distance and an adaptive kernel‑based similarity matrix to achieve geometry‑aware pruning of candidate tokens, enhancing semantic diversity without sacrificing probability or adding significant cost, making it a practical universal decoding add‑on.