深度持续学习中塑性的预测：一种理论视角
Predicting Plasticity in Deep Continual Learning: A Theoretical Perspective
AI总结：
本文从理论角度探讨深度持续学习中塑性预测问题，提出新的度量标准优化准备度，证明其在回归和分类任务中能更可靠地预测可训练性。
摘要：
深度持续学习要求模型在不重新训练的情况下适应新任务。然而，神经网络在训练过先前任务后可能失去适应新任务的能力，即塑性丧失。已有多种解释和诊断方法被提出。受
英文摘要：
Deep continual learning requires models to adapt to new tasks without retraining from scratch. However, neural networks can lose their ability to adapt to new tasks after training on previous ones, a phenomenon known as loss of plasticity. There have been several explanations and diagnostics proposed for plasticity loss. Motivated by the philosophy that "all models are wrong, but some are useful", we ask: can existing diagnostics predict a neural network's plasticity? In this work, we take a practical view to interpret plasticity as trainability, i.e., a neural network's future optimization gain on a target task. We first take a theoretical approach, showing, by constructing a few counterexamples, that some widely adopted diagnostics of plasticity, including representation rank and neural tangent kernel rank, can fail to predict the loss of trainability in both regression and classification settings. We instead propose a novel metric, called optimization readiness, which combines gradient strength and gradient reliability. We prove that optimization readiness lower bounds one-step optimization gain under standard smoothness assumptions, providing a theoretical guarantee for its predictive power. Empirically, we show that across commonly used deep continual learning settings, such as Slowly-Changing Regression and Permuted MNIST, optimization readiness more reliably ranks checkpoints by trainability than prior diagnostics, even with substantially fewer samples.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
