We introduce ExTernD (Expanded-rank Ternary Decomposition), a post-training factorization of each LLM weight matrix $A \in \mathbb{R}^{m \times n}$ into $A \approx B \operatorname{diag}(D) C$ with ternary factors $B \in \{-1,0,+1\}^{m \times k}$, $C \in \{-1,0,+1\}^{k \times n}$, and a real scale vector $D \in \mathbb{R}^k$.
The inner rank $k = \mu \min(m,n)$ is deliberately expanded beyond full rank ($\mu > 1$) to correct the quantization error of earlier components. We prove that the residual decreases monotonically in $k$ and can be driven below any $\varepsilon > 0$: ExTernD approaches bf16 accuracy arbitrarily closely, which no ternary scheme with a fixed plane count can achieve.
Memory and compute scale continuously with $\mu$, and factor sparsity scales continuously with a threshold $\tau$, allowing accuracy targets to be hit exactly rather than rounded to the next bit-width. ExTernD matches Q4_K's per-matrix accuracy at 5.2-5.5 effective bpw (5.1-5.5 with importance weighting) on Gemma-4-E2B and Qwen3.5-4B, and a full Qwen3.5-4B conversion at $\mu = 3$ reaches 10.10 wikitext-2 perplexity against 9.78 for bf16 (+3.2%), placing it near the Q4_K/Q5_K accuracy band at ~5.7 effective bpw.
Blogger's Review: The innovation of ExTernD lies in its expanded inner rank design, effectively correcting quantization errors and achieving high accuracy close to bf16. This method has significant practical value in post-training quantization for LLMs, especially in resource-constrained scenarios, demonstrating strong flexibility and adaptability. Its performance also provides new insights for future quantization technologies.