Introduction
Continual learning typically relies on post-hoc mechanisms such as replay, elastic regularization, or distillation. This work argues that forgetting should be directly modeled as interference between tasks. In the frozen-feature regime, forgetting from learning a new task is exactly the interference energy induced on the old task.
Calculation of Interference Energy
In deep networks, the same quantity can be recovered through path-averaged curvature with minimal additional forward passes. When task supports are disjoint, forgetting can be structurally eliminated; however, when task supports overlap in conflicting directions, a non-zero distortion floor is unavoidable.
Task-Aware Orthogonalization
The same geometry optimally merges models through task-aware orthogonalization. Based on this analysis, we derive Interference-Gated Functional Allocation (IGFA), a replay-free, Fisher-free method that shares directions when tasks align and protects them when they conflict.
Performance Results
Across benchmarks, IGFA achieves lossless retention when tasks are structurally separable and moves unavoidable cost from irreversible forgetting into deferred but recoverable plasticity when they are not. It matches the strongest replay-free structural baselines on dissimilar-task streams and improves on unconditional projection when similarity makes transfer worth preserving.
Blogger's Review: The introduction of the IGFA method offers a novel perspective on continual learning, effectively addressing the issue of interference-induced forgetting between tasks. This research not only advances the field of continual learning but also lays a strong foundation for future studies in related areas.