Understanding the global geometry of neural‑network loss functions is difficult because of their extremely high dimensionality. Symmetries that stem from network architectures and learning objectives offer a natural entry point for analysis. While many studies have highlighted these symmetries, a systematic characterization of them and of the resulting critical orbits remains lacking. In this work we introduce a topological symmetry index derived from the family of equivariant degree theories, applicable to arbitrary architectures and capable of handling degenerate critical orbits. The index detects the isotropy subgroup of each critical point and quantifies its net topological contribution, formally expressed as $$\mathrm{Ind}_G(f, U)=\sum_{[x]\in \mathcal{O}} \frac{1}{|G_x|}$$ where $G_x$ is the isotropy group of $x$ and $\mathcal{O}$ denotes the set of critical orbits. Several examples—including softplus MLPs, Transformers, and equivariant neural networks—illustrate the concrete computation of the index.
Review