经验回放中的非均匀记忆采样
Non-Uniform Memory Sampling in Experience Replay
AI总结：
本文质疑经验回放默认均匀采样记忆样本的最优性，通过随机生成50种非均匀采样权重并跨缓冲区、模型和数据集比较，证明始终存在显著优于均匀基线的分布。
摘要：
持续学习是在数据分布随时间变化的一系列任务上训练机器学习模型的过程。该场景中一个众所周知的障碍是灾难性遗忘，即模型在学习新任务时，会大幅丧失在先前已学任务上的性能。缓解这一问题的一种流行策略是经验回放，即将旧样本的一个子集存储在记忆缓冲区中，并与新数据一起回放。尽管持续学习的进展集中于应存储哪些样本以及如何将其纳入训练损失，但大多数方法默认假设从该缓冲区采样是均匀的。我们对均匀采样必然最优这一假设提出质疑。我们开展了一项实验，其中记忆缓冲区在每次试验中都以相同方式更新，但每个已存储样本的回放概率会在不同试验之间依据不同的随机权重分布而变化。具体而言，我们为每次试验生成50种不同的非均匀采样概率权重，并将其最终准确率与均匀采样基线进行比较。我们发现，在多种缓冲区大小、模型和数据集上，始终至少存在一种分布显著优于基线。这些结果表明，更具原则性的自适应回放策略可能带来进一步提升。我们讨论了利用这一洞见如何能够启发持续学习中非均匀记忆采样的新研究，以更好地缓解灾难性遗忘。支持本研究的代码可在 https://github.com/DentonJC/memory-sampling 获取。
英文摘要：
Continual learning is the process of training machine learning models on a sequence of tasks where data distributions change over time. A well-known obstacle in this setting is catastrophic forgetting, a phenomenon in which a model drastically loses performance on previously learned tasks when learning new ones. A popular strategy to alleviate this problem is experience replay, in which a subset of old samples is stored in a memory buffer and replayed with new data. Despite continual learning advances focusing on which examples to store and how to incorporate them into the training loss, most approaches assume that sampling from this buffer is uniform by default. We challenge the assumption that uniform sampling is necessarily optimal. We conduct an experiment in which the memory buffer updates the same way in every trial, but the replay probability of each stored sample changes between trials based on different random weight distributions. Specifically, we generate 50 different non-uniform sampling probability weights for each trial and compare their final accuracy to the uniform sampling baseline. We find that there is always at least one distribution that significantly outperforms the baseline across multiple buffer sizes, models, and datasets. These results suggest that more principled adaptive replay policies could yield further gains. We discuss how exploiting this insight could inspire new research on non-uniform memory sampling in continual learning to better mitigate catastrophic forgetting. The code supporting this study is available at $\href{https://github.com/DentonJC/memory-sampling}{https://github.com/DentonJC/memory-sampling}$.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
