In the field of Continual Learning (CL), the long-standing goal has been to mitigate catastrophic forgetting. This objective rests on a pervasive, often unstated assumption: that a lifelong learner should approximate the Joint-Task Learning (JTL) solution and retain all previously acquired knowledge. We challenge this retention-centered premise, arguing that in non-stationary environments, prioritizing retention can hinder real-time adaptation. We shift the focus to the Average Lifelong Error (ALE), formalizing CL as an online optimization problem governed by the interaction between environmental and learning dynamics.
We introduce Transfer Efficiency as a quantitative measure of the tension between Instability, the bias inherited from conflicting past experiences, and Transient Error, the optimization cost of learning new tasks from scratch. Under mild convergence conditions, holding across linear and neural network models, this decomposition yields a Critical Task Duration: a closed-form threshold beyond which historical knowledge transitions from a warm-start advantage to an optimization liability whenever retention induces a positive stationary bias.
We validate these theoretical predictions on continual image classification and reinforcement learning benchmarks. Finally, by connecting continual learning to the online learning framework of predictable sequences, we show that JTL is merely one instance of a broader family of objectives, and we propose a new general class of continual learning algorithms, termed Predictive Continual Learning. Predictive CL algorithms optimize expected future performance under an explicit, dynamically updated model of future tasks. As a proof of concept, we analyze a Window algorithm that interpolates between JTL and Independent-Task Learning (ITL), outperforming both under controlled distributional drift.