在数据节食下的持续学习
Continual Learning on a Data Diet
AI总结：
本文探索在持续学习中通过核心集选择技术从重要样本学习，实证表明选择性样本能提升增量准确性、知识保留和表示质量。
摘要：
持续学习（CL）方法通常从所有可用数据中学习。然而，人类认知并非如此，人类认知会高效地聚焦于关键经验，同时忽略冗余信息。类似地，数据集中并非所有数据点都具有同等的潜力；有些数据点可能比其他数据点更具信息量。这种差异可能会显著影响性能，因为样本的质量和数量都会直接影响模型的泛化能力和效率。受此启发，我们探索了从重要样本中学习的潜力，并提出了一项实证研究，用于在持续学习背景下评估核心集选择技术，以推动这一尚未探索领域的研究。我们在不断增加所选样本数量的情况下训练不同的持续学习器，并通过揭示驱动其稳定性-可塑性平衡改善的底层机制来研究学习-遗忘动态。我们提出了若干重要观察结果：从选择性挑选的样本中学习（i）增强了增量准确性，（ii）改善了对先前任务的知识保留，以及（iii）细化了所学表示。该分析有助于更深入地理解持续学习场景中的选择性学习策略。
英文摘要：
Continual Learning (CL) methods usually learn from all available data. However, this is not the case in human cognition which efficiently focuses on key experiences while disregarding the redundant information. Similarly, not all data points in a dataset have equal potential; some can be more informative than others. This disparity may significantly impact the performance, as both the quality and quantity of samples directly influence the model's generalizability and efficiency. Drawing inspiration from this, we explore the potential of learning from important samples and present an empirical study for evaluating coreset selection techniques in the context of CL to stimulate research in this unexplored area. We train different continual learners on increasing amounts of selected samples and investigate the learning-forgetting dynamics by shedding light on the underlying mechanisms driving their improved stability-plasticity balance. We present several significant observations: learning from selectively chosen samples (i) enhances incremental accuracy, (ii) improves knowledge retention of previous tasks, and (iii) refines learned representations. This analysis contributes to a deeper understanding of selective learning strategies in CL scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
