Neural networks trained on modular addition often develop internal representations structured by Fourier modes, enabling exact generalisation. Previous studies identified these Fourier circuits but left open how gradient descent selects them from the data distribution. We introduce probability signatures, which express leading gradient interactions via conditional statistics of the training set. For modular addition, the signatures are cyclic shift operators $S$, diagonalised by the discrete Fourier transform (DFT), yielding approximately decoupled Fourier‑mode dynamics: $$\hat{w}_k' \approx \lambda_k \hat{w}_k,$$ where $\lambda_k$ is determined by the conditional covariance of the data. This explains the emergence of Fourier sparsity, frequency matching and phase alignment.\ A puzzling phenomenon under label noise is that corrupted examples can exhibit faster early loss reduction than clean ones, despite lacking a coherent generalisation rule. We show that noise increases the probability of conditional label collisions, strengthening early reinforcement of shared coordinates and thus accelerating loss decrease.\ The framework extends to other operators. Using XOR as a test case, experiments revealed the predicted frequency, confirming the theory.\
Review: The paper maps gradient interactions to conditional statistics via probability signatures, uncovering why Fourier modes naturally arise in modular addition learning and resolving the early‑loss‑drop paradox under noise, offering a versatile tool for analysing discrete operators.