In this article, we derive four memory-optimal inference artifacts for transformer attention using the Mathematics of Arrays (MoA), each directly stemming from the Denotational Normal Form (DNF) with a fixed query-row index.
-
Single-Query Decode DNF: The $\tau$-reduction algebraically eliminates the $K^{\top}$ buffer, achieving $(d_k + nd_k + nd_v + d_v) \times 4 \text{B}$ of Dynamic Random Access Memory (DRAM) traffic, with numerical verification error bounded by $\|\text{err}\|_{\leq} \leq 2 \times 10^{-7}$.
-
C/OpenACC GPU Kernel: Features Operational Normal Form (ONF) stride arithmetic and hardware-coalesced memory access, with verified error of $\|\text{err}\|_{\infty} = 0$ (exact IEEE-754 floating-point arithmetic).
-
Multi-Step KV-Cache: Achieves $O(d_k + d_v)$ per-step append via MoA concatenation.
-
Grouped-Query Attention (GQA) and Multi-Query Attention (MQA): Derived through $\tau$-selection, proving a reduction of $\frac{h_q}{h_{kv}}$ in KV traffic.
All programs are verified against PyTorch's scaled_dot_product_attention.