Abstract
The signed integer alphabet contains one more negative representable value than positive. However, the standard symmetric integer quantizer conventionally fixes its scale to be strictly positive, assigning this extra representable value to the negative tail, which can force clipping of positive outliers. This work demonstrates that, at few-bit precision, such clipping is a non-trivial source of quantization error. Asymmetric quantization addresses this issue by introducing a zero point, shifting the grid towards the observed data range; however, this flexibility is known to incur a runtime penalty. For instance, in llama.cpp on an AMD EPYC(TM) "Turin" CPU, a 4-bit symmetric format uses up to 9% less memory with up to 2.45$\times$ higher throughput than its asymmetric counterpart. We highlight signed symmetric quantization as a third option that retains the runtime profile of symmetric quantization without the penalty of the asymmetric format: our signed absmax grid places the extra representable value on the dominant-outlier tail through a principled and lightweight sign selection rule while keeping the zero point at zero.
Our theoretical analysis offers two main results. First, we establish the signed absmax grid as conditionally bound-optimal on $\ell_2$ quantization error, showing that the condition holds for 88-99% of weight groups across pre-trained large language models (LLMs) at low bit widths. Second, we demonstrate that negating the scale of a standard symmetric quantizer is analytically equivalent to a unit zero point shift on the same signed integer alphabet. We empirically validate our proposal on models from the Qwen3, Qwen3.5, and Llama3 families, observing improvements in perplexity and downstream few-shot accuracy over the standard unsigned symmetric quantizer at no extra inference cost.
Blogger's Review: This paper cleverly addresses the common clipping issue in few-bit integer quantization through the introduction of signed symmetric quantization, maintaining performance while reducing quantization error. This method shows promising potential in practical applications for large language models, warranting further exploration and application.