回望过去：在持续学习中改进生成回放的更好知识保留
Looking through the past: better knowledge retention for generative replay in continual learning
AI总结：
本文针对持续学习中生成回放难以处理复杂数据的问题，提出潜在空间蒸馏、潜在匹配和生成循环三项改进，显著提升了生成回放在复杂场景下的知识保留能力。
摘要：
在这项工作中，我们改进了持续学习场景下的生成回放方法，使其在具有挑战性的场景中表现良好。当前的生成回放方法通常在小型且简单的数据集上进行基准测试，因为它们不足以生成具有更多类别的更复杂数据。我们注意到，在基于VAE的生成回放中，这可能归因于当映射到潜在空间时，生成的特征与原始特征相距甚远。因此，我们提出了三项修改，使模型能够学习和生成复杂数据。更具体地说，我们在当前模型和先前模型之间的潜在空间中引入蒸馏，以减少特征漂移。此外，提出了一种针对重建数据和原始数据的潜在匹配，以改善生成特征的对齐。进一步地，基于重建在保留知识方面更好的观察，我们增加了通过先前训练模型对生成样本进行循环的过程，使其更接近原始数据。我们的方法在各种场景中优于其他生成回放方法。代码可在 https://github.com/valeriya-khan/looking-through-the-past 获取。
英文摘要：
In this work, we improve the generative replay in a continual learning setting to perform well on challenging scenarios. Current generative rehearsal methods are usually benchmarked on small and simple datasets as they are not powerful enough to generate more complex data with a greater number of classes. We notice that in VAE-based generative replay, this could be attributed to the fact that the generated features are far from the original ones when mapped to the latent space. Therefore, we propose three modifications that allow the model to learn and generate complex data. More specifically, we incorporate the distillation in latent space between the current and previous models to reduce feature drift. Additionally, a latent matching for the reconstruction and original data is proposed to improve generated features alignment. Further, based on the observation that the reconstructions are better for preserving knowledge, we add the cycling of generations through the previously trained model to make them closer to the original data. Our method outperforms other generative replay methods in various scenarios. Code available at https://github.com/valeriya-khan/looking-through-the-past.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
