通过基于原型核心集的时空梯度匹配提升异构联邦持续学习的泛化能力
Improving Generalization in Heterogeneous Federated Continual Learning via Spatio-Temporal Gradient Matching with Prototypical Coreset
AI总结：
针对异构联邦持续学习中生成重放易遗忘、客户端任务差异导致过拟合的问题，提出STAMP方法，通过模型无关原型筛选、时空梯度匹配及原型近似梯度提升性能，优于现有基线。
摘要：
联邦持续学习（Federated Continual Learning, FCL）近年来已成为关键研究领域，因为分布式客户端的数据通常以流的形式到达，需要进行序列学习。本文探索了一种更具实用性和挑战性的FCL场景：客户端可能拥有不相关甚至相互冲突的数据与任务。在该场景下，统计异构性和数据噪声会产生虚假相关性，导致特征学习存在偏差以及灾难性遗忘问题。现有FCL方法通常采用生成重放技术构建过往任务的伪数据集，但生成重放本身存在灾难性遗忘问题，且客户端之间存在任务差异，会导致FCL出现过拟合。为应对上述挑战，我们提出了一种名为STAMP（Spatio-Temporal grAdient Matching with network-free Prototype，无网络原型时空梯度匹配）的新方法。我们的贡献分为三方面：1）提出一种模型无关的方法，用于确定能在原型网络中有效构成原型的样本子集，使其能应对持续学习的各类挑战；2）引入时空梯度匹配方法，在客户端（时间维度）和服务器端（空间维度）同时应用，以缓解灾难性遗忘和数据异构性问题；3）利用原型近似逐任务梯度，提升客户端侧的梯度匹配效果。大量实验表明，我们的方法性能优于现有基线方法。
英文摘要：
Federated Continual Learning (FCL) has recently emerged as a crucial research area, as data from distributed clients typically arrives as a stream, requiring sequential learning. This paper explores a more practical and challenging FCL setting, where clients may have unrelated or even conflicting data and tasks. In this scenario, statistical heterogeneity and data noise can create spurious correlations, leading to biased feature learning and catastrophic forgetting. Existing FCL approaches often use generative replay to create pseudo-datasets of previous tasks. However, generative replay itself suffers from catastrophic forgetting and task divergence among clients, leading to overfitting in FCL. Existing FCL approaches often use generative replay to create pseudo-datasets of previous tasks. However, generative replay itself suffers from catastrophic forgetting and task divergence among clients, leading to overfitting in FCL. To address these challenges, we propose a novel approach called Spatio-Temporal grAdient Matching with network-free Prototype (STAMP). Our contributions are threefold: 1) We develop a model-agnostic method to determine subset of samples that effectively form prototypes when using a prototypical network, making it resilient to continual learning challenges; 2) We introduce a spatio-temporal gradient matching approach, applied at both the client-side (temporal) and server-side (spatial), to mitigate catastrophic forgetting and data heterogeneity; 3) We leverage prototypes to approximate task-wise gradients, improving gradient matching on the client-side. Extensive experiments demonstrate our method's superiority over existing baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
