Muon has recently emerged as a strong optimizer for large-scale deep learning, reshaping gradient updates through approximate orthogonalization and reportedly outperforming Adam and AdamW in large language model training. Its empirical success has motivated a growing body of theoretical work interpreting Muon as steepest descent under the spectral norm. However, it remains unclear which advantages of Muon arise from its update rule versus those that are artifacts of the scale, architecture, and data of modern deep networks. This work isolates the optimizer from confounding factors by studying Muon on a simple, well-understood, and spectrally structured problem: low-rank matrix factorization. Through controlled comparisons against carefully tuned adaptive baselines, we find that Muon does not consistently outperform AdamW in this setting, and several previously reported advantages are sensitive to hyperparameter choices. Our results provide a more nuanced view of when spectrum-aware orthogonalization is beneficial and argue for evaluating modern optimizers on controlled problems in addition to end-to-end benchmarks.
Blogger's Review: The emergence of Muon as an optimizer presents an intriguing case for further exploration in both theoretical and practical contexts. This study highlights that performance can vary significantly under different conditions, underscoring the importance of experimental design and hyperparameter tuning.