arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.02596cs.LGcs.AI
收藏

慢而稳者胜:用兔与龟网络维持可塑性

Slow and Steady Wins the Race: Maintaining Plasticity with Hare and Tortoise Networks

Konkuk University(建国大学);Deepmind(深度思维)

Hojoon Lee, Hyeonseo Cho, Hyunseung Kim, Donghu Kim, Dugki Min, Jaegul Choo, Clare Lyle

AI总结:

本文提出 Hare & Tortoise 方法,通过快速适应的 Hare 网络与逐步整合知识的 Tortoise 网络周期性同步,在保留先验知识的同时维持神经网络可塑性,从而提升 Atari-100k 基准上强化学习算法的泛化性能。

摘要:

本研究探讨神经网络泛化能力的丧失问题,重新审视 Ash & Adams 的热启动实验。我们的实证分析表明,旨在通过维持可训练性来增强可塑性的常见方法对泛化能力的提升有限。虽然重新初始化网络可能有效,但也存在丢失有价值先验知识的风险。为此,我们受大脑互补学习系统启发,提出 Hare & Tortoise 方法。Hare & Tortoise 由两个组件组成:Hare 网络,类似于海马体,快速适应新信息;Tortoise 网络,类似于新皮层,逐步整合知识。通过周期性将 Hare 网络重新初始化为 Tortoise 网络的权重,我们的方法在保留通用知识的同时维持可塑性。Hare & Tortoise 能够有效保持网络的泛化能力,从而在 Atari-100k 基准上改进先进的强化学习算法。代码可在 https://github.com/dojeon-ai/hare-tortoise 获取。

英文摘要:

This study investigates the loss of generalization ability in neural networks, revisiting warm-starting experiments from Ash & Adams. Our empirical analysis reveals that common methods designed to enhance plasticity by maintaining trainability provide limited benefits to generalization. While reinitializing the network can be effective, it also risks losing valuable prior knowledge. To this end, we introduce the Hare & Tortoise, inspired by the brain's complementary learning system. Hare & Tortoise consists of two components: the Hare network, which rapidly adapts to new information analogously to the hippocampus, and the Tortoise network, which gradually integrates knowledge akin to the neocortex. By periodically reinitializing the Hare network to the Tortoise's weights, our method preserves plasticity while retaining general knowledge. Hare & Tortoise can effectively maintain the network's ability to generalize, which improves advanced reinforcement learning algorithms on the Atari-100k benchmark. The code is available at https://github.com/dojeon-ai/hare-tortoise.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑