持续学习中遗忘的数据依赖与预言界
Data-dependent and Oracle Bounds on Forgetting in Continual Learning
AI总结：
本文针对持续学习中的遗忘问题，为无样本回放方法提出数据依赖上界和吉布斯后验预言界，并据此设计算法，在多个持续学习问题上验证了界的紧致性与实用性。
摘要：
在持续学习中，知识必须在任务之间得到保留和复用，既要保持对未来任务的良好迁移，又要尽量减少对已学任务的遗忘。尽管针对这一场景已经设计出若干实用算法，但旨在一般设定下量化和界定遗忘程度的理论工作仍然很少。对于无样本回放方法，我们提供了无论模型和算法选择如何均适用的数据依赖上界，以及针对吉布斯后验分布的预言界。我们基于这些界推导出一种算法，并通过实验证明，在若干持续学习问题和算法上，我们的方法能够给出关于遗忘的紧致且实用的界。
英文摘要：
In continual learning, knowledge must be preserved and re-used between tasks, maintaining good transfer to future tasks and minimizing forgetting of previously learned ones. While several practical algorithms have been devised for this setting, there have been few theoretical works aiming to quantify and bound the degree of Forgetting in general settings. For \emph{exemplar-free} methods, we provide both data-dependent upper bounds that apply \emph{regardless of model and algorithm choice}, and oracle bounds for Gibbs posteriors. We derive an algorithm based on our bounds and demonstrate empirically that our approach yields tight and practical bounds on forgetting for several continual learning problems and algorithms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
