发表机构
Stanford University; Toyota Research Institute（斯坦福大学; 丰田研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文针对持续机器人学习的灾难性遗忘问题，提出记忆锚点方法，在重放缓冲器中保留关键经验，可降低63%高冲突任务遗忘，提升真实机器人的持续学习性能。
AI 中文摘要
部署在真实场景中的机器人策略应具备持续学习新任务且不遗忘已有行为的能力。对抗这种灾难性遗忘的常用方法是在新任务数据上进行训练，同时使用包含先前学习任务数据的重放缓冲器。尽管该缓冲器通常从所有先前经验中随机采样，但本文表明，其中一小部分经验对锚定过去性能贡献巨大，将这些经验称为记忆锚点。这些记忆锚点存在于新任务观测的表征与旧任务观测的表征发生重合的区域，即便这些任务需要冲突性动作，例如必须以新方式操纵熟悉的物体。在该区域中回放旧数据对防止过去任务知识的破坏性覆盖起到关键作用，承担了记忆锚点的核心角色。在对缓冲器采样前仅排除10%的记忆锚点，就会导致LIBERO基准套件上的灾难性遗忘增加超过4.5倍；相反，用记忆锚点丰富重放缓冲器可将高冲突任务的遗忘降低63%，并使真实机器人成功完成两个任务序列的持续学习。相关视频和额外可视化内容可在该https URL获取。
英文摘要
Robot policies deployed in the wild should have the capability to continually learn new tasks without forgetting existing behaviors. A common approach to combat such catastrophic forgetting is to train on new task data with a replay buffer of previously learned task data. Although this buffer is commonly sampled randomly from all prior experiences, we show that a small set of these experiences contributes greatly in anchoring past performance. We call these experiences Memory Anchors. We identify Memory Anchors in regions where representations of new-task observations collapse onto those of old-task observations even though the tasks require conflicting actions, like when a familiar object must be manipulated in a new way. Rehearsing old data in this region plays a key role in preventing destructive overwriting of past task knowledge, serving as this critical Memory Anchor role. Excluding only 10% Memory Anchors before sampling the buffer leads to more than a 4.5x increase in catastrophic forgetting on the LIBERO benchmark suites. Conversely, enriching the replay buffer with Memory Anchors can decrease high-conflict task forgetting by 63% and enables successful continual learning of two task sequences on a real robot. Videos and additional visualizations can be found at https://robot-adaptation.github.io/MemoryAnchors
Comments22 pages, 15 figures
