Data-Distill-Net：一种专为基于回放的持续学习定制的数据蒸馏方法
Data-Distill-Net: A Data Distillation Approach Tailored for Reply-based Continual Learning
AI总结：
针对基于回放的持续学习中记忆缓冲区容量有限及启发式选择标准导致的问题，提出Data-Distill-Net框架，通过可学习的记忆缓冲区与轻量级蒸馏模块生成软标签实现全局信息蒸馏，有效缓解了模型遗忘并取得优异结果。
摘要：
基于回放的持续学习（CL）方法假设，在少量子集上训练的模型也能有效最小化完整数据集的经验风险。这些方法维护一个记忆缓冲区，存储先前任务的采样数据子集以巩固过去知识。然而，由于记忆缓冲区容量有限以及缓冲区数据选择采用的启发式标准，该假设在实践中无法保证。为解决此问题，我们提出了一种专为CL定制的新数据集蒸馏框架，该框架维护一个可学习的记忆缓冲区，以从当前任务数据和先前记忆缓冲区中保留的累积知识中蒸馏全局信息。此外，为避免在蒸馏过程中对整个缓冲区进行参数化所带来的计算开销和过拟合风险，我们引入了一个轻量级蒸馏模块，该模块仅通过为记忆缓冲区数据生成可学习的软标签即可实现全局信息蒸馏。大量实验表明，我们的方法能取得有竞争力的结果，并在各种数据集上有效缓解遗忘。源代码将公开发布。
英文摘要：
Replay-based continual learning (CL) methods assume that models trained on a small subset can also effectively minimize the empirical risk of the complete dataset. These methods maintain a memory buffer that stores a sampled subset of data from previous tasks to consolidate past knowledge. However, this assumption is not guaranteed in practice due to the limited capacity of the memory buffer and the heuristic criteria used for buffer data selection. To address this issue, we propose a new dataset distillation framework tailored for CL, which maintains a learnable memory buffer to distill the global information from the current task data and accumulated knowledge preserved in the previous memory buffer. Moreover, to avoid the computational overhead and overfitting risks associated with parameterizing the entire buffer during distillation, we introduce a lightweight distillation module that can achieve global information distillation solely by generating learnable soft labels for the memory buffer data. Extensive experiments show that, our method can achieve competitive results and effectively mitigates forgetting across various datasets. The source code will be publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
