在线持续学习的多阶段牛顿迭代算法的统计理论
Statistical Theory of Multi-stage Newton Iteration Algorithm for Online Continual Learning
AI总结：
针对在线持续学习中的灾难性遗忘问题，提出引入随机效应且参数维度可趋于无穷大的统计框架，并开发多步牛顿迭代算法降低计算成本，理论上推导了估计量的渐近正态性。
摘要：
我们关注在线持续学习环境中处理非平稳数据流的关键挑战，其中受限的存储容量阻碍了历史数据的完整保留，导致在序列任务训练期间发生灾难性遗忘。为了更有效地分析和解决持续学习中的灾难性遗忘问题，我们从统计视角提出了一种新颖的持续学习框架。我们的方法在所有模型参数中引入随机效应，并允许参数维度趋于无穷大，为持续学习问题提供了一般性表述。为了高效处理流数据，我们开发了一种 Multi-step Newton Iteration 算法，该算法通过减轻矩阵求逆的负担，在某些场景下显著降低了计算成本。在理论上，我们推导了估计量的渐近正态性，从而支持后续的统计推断。通过合成数据实验和两个真实数据集分析的全面验证，证明了所提方法的有效性。
英文摘要：
We focus on the critical challenge of handling non-stationary data streams in online continual learning environments, where constrained storage capacity prevents complete retention of historical data, leading to catastrophic forgetting during sequential task training. To more effectively analyze and address the problem of catastrophic forgetting in continual learning, we propose a novel continual learning framework from a statistical perspective. Our approach incorporates random effects across all model parameters and allows the dimension of parameters to diverge to infinity, offering a general formulation for continual learning problems. To efficiently process streaming data, we develop a Multi-step Newton Iteration algorithm that significantly reduces computational costs in certain scenarios by alleviating the burden of matrix inversion. Theoretically, we derive the asymptotic normality of the estimator, enabling subsequent statistical inference. Comprehensive validation through synthetic data experiments and two real datasets analyses demonstrates the effectiveness of our proposed method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
