Neural networks frequently develop representations where individual neurons contribute to multiple distinct functions, leading to the common belief that they lack modular organization. We introduce a formally nested hierarchy of modularity:
- Unit-level: distinct computations occupy disjoint neurons;
- Orthogonal: asks whether a rotation can reveal functionally independent subspaces;
- Linear: asks whether a bounded linear map can separate task‑relevant computations into independent subspaces.
A unified framework is built to learn decompositions at each level and to evaluate them causally via selective ablation. Across vision models, language models, and multi‑task recurrent neural networks (RNNs), we find that even when networks lack unit‑level modularity, they often exhibit clear functional modularity at the subspace level.
In vision models, content and style subspaces dissociate under subspace ablation; in language models, syntax and semantics similarly separate, despite overlapping neuron populations. Alignment with brain data shows that content subspaces better predict higher‑level visual cortex activity, while style subspaces align with lower‑level visual areas; semantic subspaces outperform syntactic ones in predicting language‑cortex responses.
For previously reported non‑modular multi‑task RNNs, our decomposition recovers sharp double dissociations between tasks, demonstrating that specific computations can be isolated even without specialized single neurons.
These results indicate that the absence of unit‑level modularity often coexists with pronounced subspace‑level modularity, enabling targeted interventions on particular computations and offering broad implications for mechanistic interpretability and for understanding functional organization in both biological and artificial neural systems.
Review: This work uncovers hidden subspace modularity in deep networks, providing a powerful lens for interpreting and manipulating complex models.