利用轻量级生成器实现内存高效的持续学习
Leveraging Lightweight Generators for Memory Efficient Continual Learning
AI总结：
本文提出基于奇异值分解的轻量级生成器，用于增强A-GEM和Experience Replay等持续学习方法，以最小化内存占用并显著提高平均准确率。
摘要：
灾难性遗忘可以通过在内存中保留之前任务的所有数据来轻易缓解。因此，在最大化相关信息量的同时最小化内存占用，对于持续学习的挑战至关重要。本文旨在减少基于内存的持续学习算法所需的内存。我们探索了提取最少信息同时最大程度缓解遗忘的选项。我们提出使用基于奇异值分解的轻量级生成器来增强现有的持续学习方法，如A-GEM和Experience Replay。这些生成器需要最少的内存，同时效果最大化。它们不需要训练时间，只需一个线性时间的拟合步骤，并能从少量数据样本中有效捕获分布。根据数据集和网络架构的不同，我们的结果显示，与原始方法相比，平均准确率显著提高。我们的方法在最小化基于内存的持续学习算法的内存占用方面显示出巨大潜力。
英文摘要：
Catastrophic forgetting can be trivially alleviated by keeping all data from previous tasks in memory. Therefore, minimizing the memory footprint while maximizing the amount of relevant information is crucial to the challenge of continual learning. This paper aims to decrease required memory for memory-based continuous learning algorithms. We explore the options of extracting a minimal amount of information, while maximally alleviating forgetting. We propose the usage of lightweight generators based on Singular Value Decomposition to enhance existing continual learning methods, such as A-GEM and Experience Replay. These generators need a minimal amount of memory while being maximally effective. They require no training time, just a single linear-time fitting step, and can capture a distribution effectively from a small number of data samples. Depending on the dataset and network architecture, our results show a significant increase in average accuracy compared to the original methods. Our method shows great potential in minimizing the memory footprint of memory-based continual learning algorithms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
