Abstract
Brain field potentials are scale-free, with their power spectra following a $1/f^{\beta}$ law, where the aperiodic exponent $\beta$ reflects cortical state, particularly shifts in sleep depth. We investigate whether a transformer endowed with an explicit renormalization-group (RG) inductive bias—the RG-Flow Transformer—has an advantage over a parameter-matched vanilla transformer on extit{real, scarce} EEG data.
Using the PhysioNet Sleep-EDF corpus with a strict leakage-free by-subject hold-out, we benchmarked RG-Flow against a param-matched vanilla transformer and a hierarchy-only ablation on 5-class AASM sleep staging. We also swept the per-subject data budget to search for the inductive-bias crossover predicted when data is scarce, and tested whether RG-Flow's learned $\gamma$ tracks the measured spectral exponent $\beta$, a characteristic absent in the vanilla model.
Across 5 subjects and 5 seeds under leave-one-subject-out cross-validation, RG-Flow and the vanilla transformer were statistically indistinguishable on 5-class staging (77.3% vs 77.0% accuracy; paired $p=0.294$), and the predicted scarce-data crossover did not appear: the vanilla model was numerically ahead at every data-limited budget. The key difference between the models was interpretability—RG-Flow recovers the continuous spectral exponent out-of-sample ($\beta$-recovery $R^2 = 0.416$), a capability the vanilla architecture lacks.
Blogger's Review: The RG-Flow Transformer showcases impressive interpretability when handling scarce EEG data, achieving comparable accuracy to traditional models. Its ability to recover spectral exponents offers valuable insights into neural signals, indicating promising avenues for further exploration in other domains.