NeFut Logo NeFut
中 Admin Login

[CS.AI] "Close enough?" An Investigation into Multiplication Errors in LLMs

Published at: 2026-10-31 22:00 Last updated: 2026-11-02 00:48
#AI #Machine Learning #LLM

Large language models (LLMs) often make mistakes on multi‑digit arithmetic, yet the internal mechanisms behind these failures are not well understood. Prior work shows that addition errors follow a noisy quantisation model centred on carry boundaries, and it remains open whether multiplication adheres to the same framework.\ \ In this study we benchmarked multiplication on several small base LLMs using 16,991 problems with varied operand lengths. The results reveal that multiplication errors do not match the addition‑style noisy quantisation pattern. Instead, a novel error regime emerges in small Qwen models: errors in lower‑order columns tend to preserve parity, while higher‑order digit errors shift toward off‑by‑one carry failures.\ \ These findings suggest that arithmetic computation in LLMs relies on distinct, multi‑periodic internal representations where parity is tracked more robustly than exact digit identity, offering new mechanistic insight into numerical reasoning in transformers.\ \ Review

Original Source: https://arxiv.org/abs/2610.38634

[h] Back to Home