arXiv 2505.122452025-05-18cs.LGcs.AI

AFCL：面向非独立同分布数据时空不变性的解析型联邦持续学习

AFCL: Analytic Federated Continual Learning for Spatio-Temporal Invariance of Non-IID Data

Peking University, China（北京大学）；South China University of Technology, China（华南理工大学）；Central South University, China（中南大学）；Beijing Normal University, China（北京师范大学）；University of Southampton, UK（南安普顿大学）；University of Maryland, Baltimore County, USA（马里兰大学巴尔的摩县分校）

Jianheng Tang, Huiping Zhuang, Jingyu He, Run He, Jingchao Wang, Kejia Fan, Anfeng Liu, Tian Wang, Leye Wang, Zhanxing Zhu, Shanghang Zhang, Houbing Herbert Song, Yunhuai Liu

备注：: 23 pages, 5 figures, 5 tables

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对联邦持续学习中数据时空异质性导致的灾难性遗忘问题，提出无梯度的解析型联邦持续学习方法AFCL，通过闭式解实现数据时空不变性，性能优于现有最优基线。

摘要：

联邦持续学习（Federated Continual Learning，FCL）使分布式客户端能够在动态现实场景中，基于在线任务流协同训练全局模型。然而，现有FCL方法同时面临分布式客户端间的空间数据异质性，以及在线任务间的时间数据异质性的挑战。这种数据异质性会严重降低模型性能，引发严重的局部知识与过往知识的时空灾难性遗忘问题。\n本文指出，该问题的根源在于梯度对非独立同分布（non-IID）数据固有的脆弱性与敏感性。为从根本上解决这一问题，我们提出一种无梯度方法——解析型联邦持续学习（Analytic Federated Continual Learning，AFCL），该方法从冻结的提取特征中推导解析（即闭式）解。在本地训练阶段，AFCL支持单轮次学习，每个客户端仅需执行轻量的前向传播过程。在全局聚合阶段，服务器可通过单轮聚合递归且高效地更新全局模型。\n理论分析验证，AFCL实现了非独立同分布数据的时空不变性。这一理想特性意味着，无论数据在本地客户端与在线任务间的分布有多么异质，AFCL的聚合模型都能保持不变，且与集中式联合学习的模型完全一致。大量实验表明，在各类基准数据集与设置下，AFCL的性能始终优于现有最优基线方法。

英文摘要：

Federated Continual Learning (FCL) enables distributed clients to collaboratively train a global model from online task streams in dynamic real-world scenarios. However, existing FCL methods face challenges of both spatial data heterogeneity among distributed clients and temporal data heterogeneity across online tasks. Such data heterogeneity significantly degrades the model performance with severe spatial-temporal catastrophic forgetting of local and past knowledge. In this paper, we identify that the root cause of this issue lies in the inherent vulnerability and sensitivity of gradients to non-IID data. To fundamentally address this issue, we propose a gradient-free method, named Analytic Federated Continual Learning (AFCL), by deriving analytical (i.e., closed-form) solutions from frozen extracted features. In local training, our AFCL enables single-epoch learning with only a lightweight forward-propagation process for each client. In global aggregation, the server can recursively and efficiently update the global model with single-round aggregation. Theoretical analyses validate that our AFCL achieves spatio-temporal invariance of non-IID data. This ideal property implies that, regardless of how heterogeneous the data are distributed across local clients and online tasks, the aggregated model of our AFCL remains invariant and identical to that of centralized joint learning. Extensive experiments show the consistent superiority of our AFCL over state-of-the-art baselines across various benchmark datasets and settings.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。