Quantizing weights below four bits dramatically reduces memory and bandwidth while keeping perplexity tolerable, but it often sacrifices factual knowledge and task accuracy. Existing ternary weight methods usually rely on training from scratch or long fine‑tuning. RoleQ introduces a quantization framework that assigns scales according to the functional roles of Transformer units instead of the layout of the weight matrix. Vectors that read from the residual stream use one scale per output unit; vectors that write to it use one scale per input unit. This rule is structural, requiring no search, calibration, or extra state, and it works on both integer and ternary grids. In a rotated basis it is equivalent to a per‑layer axis search with half the solver effort; in ternary it cuts perplexity by up to half compared with layout‑based placement, covering model sizes from 0.6B to 14B, whereas per‑row scales drop MMLU to chance level. A lightweight post‑training pipeline can convert an existing checkpoint to $1.59$ bits per weight in a few GPU‑hours, costing only a fraction of learned ternarization, and it outperforms all post‑training ternary methods at comparable or lower memory‑resident bits on perplexity, zero‑shot accuracy, and MMLU—especially on knowledge tasks. Adding a sparse second ternary plane placed by per‑unit Fisher saliency keeps matrix multiplication free and recovers the performance most lost by ternarization: at 8B, adding less than a bit restores about half the MMLU, far ahead of random or Hessian‑based placement at equal bits.
Review: RoleQ demonstrates that a role‑aligned scaling strategy can achieve ultra‑low‑bit quantization without compromising inference quality, offering a practical path for post‑training ternarization of large language models.