深度学习中持续学习之外的遗忘全面综述
A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning
AI总结：
本文综述了持续学习之外的深度学习遗忘现象，指出遗忘在生成模型和联邦学习等领域普遍存在，并强调其不仅是挑战，在隐私保护等场景下也具潜在优势，旨在促进跨领域解决方案与新策略的开发。
摘要：
遗忘是指先前获取知识的丢失或退化。尽管现有的关于遗忘的综述主要集中在持续学习上，但遗忘是深度学习其他多个研究领域中普遍存在的现象。由于生成器偏移，遗忘在生成模型等领域中表现出来；由于客户端间异构的数据分布，遗忘在联邦学习等领域中表现出来。解决遗忘包含若干挑战，包括平衡旧任务知识的保留与新任务的快速学习、管理具有冲突目标的任务干扰，以及防止隐私泄露等。此外，现有的多数持续学习综述隐含假设遗忘总是有害的。相反，我们的综述认为遗忘是一把双刃剑，在某些情况下（如隐私保护场景）是有益且可取的。通过在更广泛的背景下探索遗忘，我们对这一现象提出了更细致的理解，并强调了其潜在优势。通过这篇全面综述，我们希望借鉴处理过遗忘的各个领域的思想和方法，揭示潜在的解决方案。通过审视超越传统边界的遗忘，我们希望鼓励开发在真实应用中减轻、利用甚至拥抱遗忘的新策略。关于各个研究领域中遗忘的论文综合列表可在 https://github.com/EnnengYang/Awesome-Forgetting-in-Deep-Learning 获取。
英文摘要：
Forgetting refers to the loss or deterioration of previously acquired knowledge. While existing surveys on forgetting have primarily focused on continual learning, forgetting is a prevalent phenomenon observed in various other research domains within deep learning. Forgetting manifests in research fields such as generative models due to generator shifts, and federated learning due to heterogeneous data distributions across clients. Addressing forgetting encompasses several challenges, including balancing the retention of old task knowledge with fast learning of new task, managing task interference with conflicting goals, and preventing privacy leakage, etc. Moreover, most existing surveys on continual learning implicitly assume that forgetting is always harmful. In contrast, our survey argues that forgetting is a double-edged sword and can be beneficial and desirable in certain cases, such as privacy-preserving scenarios. By exploring forgetting in a broader context, we present a more nuanced understanding of this phenomenon and highlight its potential advantages. Through this comprehensive survey, we aspire to uncover potential solutions by drawing upon ideas and approaches from various fields that have dealt with forgetting. By examining forgetting beyond its conventional boundaries, we hope to encourage the development of novel strategies for mitigating, harnessing, or even embracing forgetting in real applications. A comprehensive list of papers about forgetting in various research fields is available at \url{https://github.com/EnnengYang/Awesome-Forgetting-in-Deep-Learning}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
