学习遗忘：基于自适应权重衰减的持续学习
Learning to Forget: Continual Learning with Adaptive Weight Decay
AI总结：
本文提出FADE方法，通过自适应调整权重衰减率，在持续学习中平衡新旧知识的获取与保留，提升在线跟踪和流分类任务的性能。
摘要：
本文提出FADE方法，通过自适应调整权重衰减率，在持续学习中平衡新旧知识的获取与保留，提升在线跟踪和流分类任务的性能。
英文摘要：
Continual learning agents with finite capacity must balance acquiring new knowledge with retaining the old. This requires controlled forgetting of knowledge that is no longer needed, freeing up capacity to learn. Weight decay, viewed as a mechanism for forgetting, can serve this role by gradually discarding information stored in the weights. However, a fixed scalar weight decay drives this forgetting uniformly over time and uniformly across all parameters, even when some encode stable knowledge while others track rapidly changing targets. We introduce Forgetting through Adaptive Decay (FADE), which adapts per-parameter weight decay rates online via approximate meta-gradient descent. We derive FADE for the online linear setting and apply it to the final layer of neural networks. Our empirical analysis shows that FADE automatically discovers distinct decay rates for different parameters, complements step-size adaptation, and consistently improves over fixed weight decay across online tracking and streaming classification problems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
