在持续学习中解决可塑性丧失与灾难性遗忘问题
Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learning
AI总结：
本文提出基于效用的扰动梯度下降UPGD，通过保护有用单元并扰动无用单元，同时缓解持续学习中的灾难性遗忘和可塑性丧失，在流式学习与PPO强化学习实验中均表现优越。
摘要：
深度表示学习方法在持续学习中面临困难，既遭受对有用单元灾难性遗忘的困扰，也因僵化和无用的单元而丧失可塑性。尽管许多方法分别解决这两个问题，但目前只有少数方法能同时处理两者。在本文中，我们提出基于效用的扰动梯度下降（Utility-based Perturbed Gradient Descent，UPGD），作为一种用于表示持续学习的新方法。UPGD将梯度更新与扰动相结合，对更有用的单元施加较小的修改，以保护它们免于遗忘；对较无用的单元施加较大的修改，以恢复其可塑性。我们使用了一个具有挑战性的流式学习设置，其中持续学习问题包含数百个非平稳性和未知的任务边界。我们表明，许多现有方法至少受到其中一个问题的影响，主要表现为它们在各个任务上的准确率逐渐下降。相比之下，UPGD持续提升性能，并在所有问题上超越或与所有方法具有竞争力。最后，在使用PPO进行的扩展强化学习实验中，我们表明，虽然Adam在初始学习后出现性能下降，但UPGD通过同时解决这两个持续学习问题避免了这一现象。
英文摘要：
Deep representation learning methods struggle with continual learning, suffering from both catastrophic forgetting of useful units and loss of plasticity, often due to rigid and unuseful units. While many methods address these two issues separately, only a few currently deal with both simultaneously. In this paper, we introduce Utility-based Perturbed Gradient Descent (UPGD) as a novel approach for the continual learning of representations. UPGD combines gradient updates with perturbations, where it applies smaller modifications to more useful units, protecting them from forgetting, and larger modifications to less useful units, rejuvenating their plasticity. We use a challenging streaming learning setup where continual learning problems have hundreds of non-stationarities and unknown task boundaries. We show that many existing methods suffer from at least one of the issues, predominantly manifested by their decreasing accuracy over tasks. On the other hand, UPGD continues to improve performance and surpasses or is competitive with all methods in all problems. Finally, in extended reinforcement learning experiments with PPO, we show that while Adam exhibits a performance drop after initial learning, UPGD avoids it by addressing both continual learning issues.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
