从顺序到分布：对持续学习中遗忘的谱特征分析
From Order to Distribution: An Exact Operator Framework for Forgetting in Continual Learning
AI总结：
本文研究持续学习中遗忘现象，通过分析任务分布的谱结构，推导遗忘量的精确算子恒等式，并建立无条件上界，揭示遗忘率与任务分布几何性质的关系。
摘要：
持续学习中的核心挑战是遗忘，即由于连续适应新任务而失去先前学习任务的性能。尽管遗忘已被广泛研究，但严格的理论分析仍有限。本文从顺序转向分布，研究任务在i.i.d.分布下精确拟合线性回归中的遗忘机制。通过推导遗忘量的精确算子恒等式，揭示其递归谱结构。基于此恒等式，建立无条件上界，识别主导渐进行为项，并在非退化情况下刻画收敛速率。进一步将此速率与任务分布的几何性质关联，明确驱动慢或快速遗忘的因素。
英文摘要：
A central challenge in continual learning is forgetting: the loss of performance on previously learned tasks after learning new ones. Prior theory has analyzed forgetting under random orderings of fixed task collections in overparameterized linear regression. We shift the focus from task order to task distribution, asking how its structure determines forgetting. In the linear setting with a shared solution, i.i.d. task sampling, and sequential exact fitting, we derive an exact operator identity expressing historical forgetting directly in terms of the task distribution. Building on this identity, we establish an exponential decay guarantee for expected historical forgetting under every fixed task distribution in finite dimensions, characterize its asymptotic behavior, and relate decay to the distribution's coverage of observable directions. For an individual learned task, we show that subsequent tasks can collectively support recovery without exact revisits. We derive a lower bound on recovery time and construct a task distribution attaining its inverse-coverage scaling.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
