Federated learning encounters a critical challenge of data heterogeneity, adversely affecting the performance and convergence of the federated model. Various approaches have been proposed to address this issue, yet their effectiveness is still limited. Recent studies have revealed that the federated model suffers severe forgetting in local training, leading to global forgetting and performance degradation. Although the analysis provides valuable insights, a comprehensive understanding of the vulnerable classes and their impact factors is yet to be established. In this paper, we aim to bridge this gap by systematically analyzing the forgetting degree of each class during local training across different communication rounds. Our observations are: (1) Both missing and non-dominant classes suffer similar severe forgetting during local training, while dominant classes show improvement in performance. (2) When dynamically reducing the sample size of a dominant class, catastrophic forgetting occurs abruptly when the proportion of its samples is below a certain threshold, indicating that the local model struggles to leverage a few samples of a specific class effectively to prevent forgetting. Motivated by these findings, we propose a novel and straightforward algorithm called Federated Knowledge Anchor (FedKA). Assuming that all clients have a single shared sample for each class, the knowledge anchor is constructed before each local training stage by extracting shared samples for missing classes and randomly selecting one sample per class for non-dominant classes. The knowledge anchor is then utilized to correct the gradient of each mini-batch towards the direction of preserving the knowledge of the missing and non-dominant classes. Extensive experimental results demonstrate that our proposed FedKA achieves fast and stable convergence, significantly improving accuracy on popular benchmarks.
Shanghai Jiao Tong University（上海交通大学）；Meta Platforms（元平台公司）；Shanghai AI Laboratory（上海人工智能实验室）；Weizmann Institute of Science（魏茨曼科学研究所）
One of the key challenges in federated learning (FL) is local data distribution heterogeneity across clients, which may cause inconsistent feature spaces across clients. To address this issue, we propose a novel method FedFM, which guides each client's features to match shared category-wise anchors (landmarks in feature space). This method attempts to mitigate the negative effects of data heterogeneity in FL by aligning each client's feature space. Besides, we tackle the challenge of varying objective function and provide convergence guarantee for FedFM. In FedFM, to mitigate the phenomenon of…
Handling Spatial-Temporal Data Heterogeneity for Federated Continual Learning via Tail Anchor
Southwestern University of Finance and Economics（西南财经大学）；University of Electronic Science and Technology of China（电子科技大学）；WeBank（微众银行）；Southwest Jiaotong University（西南交通大学）；Hong Kong University of Science and Technology（香港科技大学）
FedADB: Class Anchor-Driven Dual-Branch Federated Learning for Mitigating Forgetting
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications（北京邮电大学网络与交换技术国家重点实验室）；National Engineering Research Center of Disaster Backup and Recovery, Beijing University of Posts and Telecommunications（北京邮电大学国家灾备工程技术研究中心）；JIUTIAN Research, China Mobile（中国移动九天研究院）；School of Cyberspace Security, Beijing University of Posts and Telecommunications（北京邮电大学网络空间安全学院）；University of Leicester（莱斯特大学）
Beyond Rigid Alignment: Graph Federated Learning via Dual Manifold Calibration
School of Computer Science and Engineering（计算机科学与工程学院）；Nanjing University of Science and Technology（南京理工大学）；Department of Computer Science（计算机科学系）；Hong Kong Baptist University（香港 Baptist 大学）；School of Automation and Intelligent Sensing（自动化与智能感知学院）
On the Push-Based Asynchronous Federated Learning: A Bias-Correction Aggregation Approach
School of Computer Technologies, RMIT University（RMIT大学计算机技术学院）；School of Science, Computing and Engineering Technologies, Swinburne University of Technology（斯威丁大学科学与工程技术学院）
Understanding Trajectory Heterogeneity in Federated World Model Learning
Wuhan University（武汉大学）；China Academy of Information and Communications Technology（中国信息通信研究院）；The University of Sydney（悉尼大学）；Tsinghua University（清华大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
BAPO: Base-Anchored Preference Optimization for Overcoming Forgetting in Large Language Models Personalization
Graduate School of AI, KAIST（韩国科学技术院人工智能研究生院）；Samsung Advanced Institute of Technology（三星先进技术研究院）；Department of Electrical and Computer Engineering, UBC（不列颠哥伦比亚大学电气与计算机工程系）
Breaking the Prototype Bias Loop: Confidence-Aware Federated Contrastive Learning for Highly Imbalanced Clients
Key Laboratory of Water Big Data Technology of Ministry of Water Resources, College of Computer Science and Software Engineering, Hohai University, Nanjing, China（水利部水大数据技术重点实验室，计算机科学与软件工程学院，河海大学，南京，中国）；Department of Computer Science, City University of Hong Kong, Hong Kong, China（计算机科学系，香港城市大学，香港，中国）；State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China（新型软件技术国家重点实验室，南京大学，南京，中国）；School of Artificial Intelligence and Information Technology, Nanjing University of Chinese Medicine, Nanjing, China（人工智能与信息技术学院，南京中医药大学，南京，中国）
