基于地图的经验回放：强化学习中灾难性遗忘的内存高效解决方案
Map-based Experience Replay: A Memory-Efficient Solution to Catastrophic Forgetting in Reinforcement Learning
AI总结：
针对深度强化学习的灾难性遗忘问题，提出基于GWR自组织网络的地图式经验回放方法，通过合并相似样本精简记忆，实现内存大幅减少且性能损失微小。
摘要：
深度强化学习智能体常遭遇灾难性遗忘问题，即在新数据上训练时会遗忘输入空间中部分区域此前找到的解决方案。回放记忆是解决该问题的常用方法，可对新旧训练样本去相关并打乱顺序。这类方法会不加选择地存储所有到来的状态转移，完全不考虑冗余性。我们提出一种受认知启发的新型回放记忆方法，该方法基于按需生长（Grow-When-Required, GWR）自组织网络，类似一种基于地图的世界心智模型。我们的方法将存储的转移组织为一个由状态节点和转移边构成的类环境模型的简洁网络，通过合并相似样本缩小内存占用并提升样本间的成对距离，进而提高每个样本的相关性。总体而言，本文表明基于地图的经验回放可大幅减少内存占用，仅带来微小的性能下降。
英文摘要：
Deep Reinforcement Learning agents often suffer from catastrophic forgetting, forgetting previously found solutions in parts of the input space when training on new data. Replay Memories are a common solution to the problem, decorrelating and shuffling old and new training samples. They naively store state transitions as they come in, without regard for redundancy. We introduce a novel cognitive-inspired replay memory approach based on the Grow-When-Required (GWR) self-organizing network, which resembles a map-based mental model of the world. Our approach organizes stored transitions into a concise environment-model-like network of state-nodes and transition-edges, merging similar samples to reduce the memory size and increase pair-wise distance among samples, which increases the relevancy of each sample. Overall, our paper shows that map-based experience replay allows for significant memory reduction with only small performance decreases.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
