arXiv 2502.142052025-02-20cs.LGcs.AI

面向异构联邦持续学习的精准遗忘

Accurate Forgetting for Heterogeneous Federated Continual Learning

Institute for Artificial Intelligence, Tsinghua University（清华大学人工智能研究院）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；The University of Auckland（奥克兰大学）；RIKEN（理化学研究所）；Hong Kong Baptist University（香港浸会大学）；DataCanvas Technology Co., Ltd.（数帆科技有限公司）；The University of Tokyo（东京大学）

Abudukelimu Wuerkaixi, Sen Cui, Jingfeng Zhang, Kunda Yan, Bo Han, Gang Niu, Lei Fang, Changshui Zhang, Masashi Sugiyama

备注：: published in ICLR 2024

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对异构联邦持续学习中客户端数据差异引发的特征学习偏差问题，提出精准遗忘概念与基于归一化流的生成重放方法，选择性利用可信旧知识，性能优于现有基线。

摘要：

近年来，联邦学习（federated learning, FL）领域的研究热度持续攀升，但客户端参与序列学习的场景仍未得到充分探索。将联邦学习与持续学习（continual learning, CL）相结合，催生了一个颇具挑战性的实际问题：联邦持续学习（federated continual learning, FCL）。现有的联邦持续学习研究主要聚焦于在与其他客户端协作的过程中，缓解持续学习的灾难性遗忘问题。我们认为，遗忘现象并非始终有害。\n本文考虑了一种更具实用性和挑战性的联邦持续学习场景：不同客户端的数据或任务可能互不相关，甚至存在对立关系。在联邦学习场景中，客户端间的统计异构性与数据噪声可能产生虚假相关性，导致特征学习出现偏差。现有持续学习策略侧重于对先前知识的完全利用，而我们的研究发现，遗忘存在偏差的信息是有益的。因此，我们提出了“精准遗忘（accurate forgetting, AF）”这一新概念，并设计了一种全新的生成重放方法，可在联邦网络中选择性利用先前知识。我们采用基于归一化流（normalizing flow）模型的概率框架，对先前知识的可信度进行量化。全面的实验验证了我们的方法相较于各基线方法的优越性。

英文摘要：

Recent years have witnessed a burgeoning interest in federated learning (FL). However, the contexts in which clients engage in sequential learning remain under-explored. Bridging FL and continual learning (CL) gives rise to a challenging practical problem: federated continual learning (FCL). Existing research in FCL primarily focuses on mitigating the catastrophic forgetting issue of continual learning while collaborating with other clients. We argue that the forgetting phenomena are not invariably detrimental. In this paper, we consider a more practical and challenging FCL setting characterized by potentially unrelated or even antagonistic data/tasks across different clients. In the FL scenario, statistical heterogeneity and data noise among clients may exhibit spurious correlations which result in biased feature learning. While existing CL strategies focus on a complete utilization of previous knowledge, we found that forgetting biased information is beneficial in our study. Therefore, we propose a new concept accurate forgetting (AF) and develop a novel generative-replay method~\method~which selectively utilizes previous knowledge in federated networks. We employ a probabilistic framework based on a normalizing flow model to quantify the credibility of previous knowledge. Comprehensive experiments affirm the superiority of our method over baselines.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。