持续岭回归中泛化性能的高维渐近分析
High-dimensional Asymptotics of Generalization Performance in Continual Ridge Regression
AI总结：
本文利用随机矩阵理论推导高维持续岭回归的渐近预测风险，刻画平均风险、反向迁移与正向迁移指标，并通过风险曲线揭示模型设定对泛化性能的影响。
摘要：
持续学习的动机在于需要适应任务和数据分布中的现实动态，同时缓解灾难性遗忘。尽管持续学习技术取得了显著进展，但对其泛化性能的理论理解仍然滞后。本文研究了高维线性模型中持续岭回归的理论性质，其中维度与每个任务中的样本量成比例。利用随机矩阵理论，我们推导出了渐近预测风险的精确表达式，从而能够刻画持续学习中泛化性能的三个评估指标：平均风险、反向迁移和正向迁移。此外，我们给出了理论风险曲线，以说明这些评估指标在整个持续学习过程中的变化趋势。我们的分析揭示了风险曲线中若干有趣的现象，展示了模型设定如何影响泛化性能。我们进行了仿真研究以验证理论发现。
英文摘要：
Continual learning is motivated by the need to adapt to real-world dynamics in tasks and data distribution while mitigating catastrophic forgetting. Despite significant advances in continual learning techniques, the theoretical understanding of their generalization performance lags behind. This paper examines the theoretical properties of continual ridge regression in high-dimensional linear models, where the dimension is proportional to the sample size in each task. Using random matrix theory, we derive exact expressions of the asymptotic prediction risk, thereby enabling the characterization of three evaluation metrics of generalization performance in continual learning: average risk, backward transfer, and forward transfer. Furthermore, we present the theoretical risk curves to illustrate the trends in these evaluation metrics throughout the continual learning process. Our analysis reveals several intriguing phenomena in the risk curves, demonstrating how model specifications influence the generalization performance. Simulation studies are conducted to validate our theoretical findings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
