通过平台期活动分析缓解灾难性遗忘
Catastrophic Forgetting Mitigation Through Plateau Phase Activity Profiling
AI总结：
针对深度神经网络中的灾难性遗忘问题，提出通过分析最终训练平台期的参数活动度来识别重要参数，在保留旧知识的同时适应新任务，实验证明该方法能有效平衡遗忘缓解与新任务性能。
摘要：
深度神经网络中的灾难性遗忘，是指在学习新任务时由于知识覆盖导致先前已学任务的性能下降。在缓解此问题的方法中，正则化技术旨在识别并约束“重要”参数以保留先前知识。在深度学习的高度非凸优化景观中，我们提出了一种新视角：在最终训练平台期跟踪参数，比在整个训练过程中监控它们更为有效。我们认为，在该平台期表现出更高活动度（运动和变异性）的参数，揭示了损失景观中相对平坦的方向，使其适合适应新任务同时保留先前知识。我们的综合实验表明，该方法在平衡灾难性遗忘缓解与对新学任务的强劲性能方面实现了卓越表现。
英文摘要：
Catastrophic forgetting in deep neural networks occurs when learning new tasks degrades performance on previously learned tasks due to knowledge overwriting. Among the approaches to mitigate this issue, regularization techniques aim to identify and constrain "important" parameters to preserve previous knowledge. In the highly nonconvex optimization landscape of deep learning, we propose a novel perspective: tracking parameters during the final training plateau is more effective than monitoring them throughout the entire training process. We argue that parameters that exhibit higher activity (movement and variability) during this plateau reveal directions in the loss landscape that are relatively flat, making them suitable for adaptation to new tasks while preserving knowledge from previous ones. Our comprehensive experiments demonstrate that this approach achieves superior performance in balancing catastrophic forgetting mitigation with strong performance on newly learned tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
