Federated low-rank adaptation (FedLoRA) has facilitated communication-efficient and privacy-preserving fine-tuning of foundation models for downstream tasks. In practical federated learning scenarios, client heterogeneity in system resources and data distributions motivates the use of heterogeneous LoRA ranks across clients. However, we identify a previously overlooked phenomenon in heterogeneous FedLoRA with SVD-based allocation, termed rank collapse, where the energy of the global update becomes concentrated in the minimum shared rank, resulting in suboptimal performance and high sensitivity to rank configurations. Through theoretical analysis, we reveal the underlying mechanism of rank collapse: a mismatch between rank-agnostic aggregation weights and rank-dependent client contributions, which systematically suppresses higher-rank updates at a geometric rate over rounds. Motivated by this insight, we propose raFLoRA, a rank-partitioned aggregation method that decomposes local updates into rank partitions and then aggregates each partition weighted by its effective client contributions. Extensive experiments across vision, language, and reasoning tasks show that raFLoRA prevents rank collapse, improves model performance, and enhances robustness across diverse heterogeneous configurations compared with strong FedLoRA baselines.
WinFLoRA: Incentivizing Client-Adaptive Aggregation in Federated LoRA under Privacy Heterogeneity
RMIT University（皇家墨尔本理工大学）；Huazhong University of Science and Technology（华中科技大学）；CSIRO’s Data61 and Responsible AI Research (RAIR) Centre（澳大利亚CSIRO数据61和负责任的人工智能研究（RAIR）中心）
FedSPM: Routing-Enabled Federated Learning under Dual Heterogeneity via Semiparametric Mixture
Institute of Statistics and Big Data, Renmin University of China（中国人民大学统计与大数据研究院）；Department of Statistics and Actuarial Science, University of Waterloo（滑铁卢大学统计与精算科学系）
HASA: Subnet Allocation for Compute-Constrained Model-Heterogeneous Federated Learning
University of Naples "Federico II"（那不勒斯腓特烈二世大学）；Queen Mary University of London（伦敦玛丽女王大学）；Brandenburg University of Technology Cottbus-Senftenberg（勃兰登堡工业大学）；Tallinn University of Technology（塔林理工大学）；University of Zanjan（赞詹大学）
Hong Kong Baptist University（香港浸会大学）；The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
In federated learning (FL), model performance typically suffers from client drift induced by data heterogeneity, and mainstream works focus on correcting client drift. We propose a different approach named virtual homogeneity learning (VHL) to directly "rectify" the data heterogeneity. In particular, VHL conducts FL with a virtual homogeneous dataset crafted to satisfy two conditions: containing no private information and being separable. The virtual dataset can be generated from pure noise shared across clients, aiming to calibrate the features from the heterogeneous clients. Theoretically, w…
The traditional approach in FL tries to learn a single global model collaboratively with the help of many clients under the orchestration of a central server. However, learning a single global model might not work well for all clients participating in the FL under data heterogeneity. Therefore, the personalization of the global model becomes crucial in handling the challenges that arise with statistical heterogeneity and the non-IID distribution of data. Unlike prior works, in this work we propose a new approach for obtaining a personalized model from a client-level objective. This further mot…
Local Gradient Regulation Stabilizes Federated Learning under Client Heterogeneity
National Key Laboratory of Parallel and Distributed Computing, College of Computer Science and Technology, National University of Defense Technology（国家并行与分布式计算实验室，计算机科学与技术学院，国防科技大学）
FedCCA: Client-Centric Adaptation against Data Heterogeneity in Federated Learning on IoT Devices
Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算机系）；Centre for Learning, Teaching, and Technology, The Education University of Hong Kong（香港教育大学学习、教学与技术中心）
Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；IBM T. J. Watson Research Center（IBM托马斯·J·沃森研究中心）；Yale University（耶鲁大学）
Federated learning (FL) algorithms usually sample a fraction of clients in each round (partial participation) when the number of participants is large and the server's communication bandwidth is limited. Recent works on the convergence analysis of FL have focused on unbiased client sampling, e.g., sampling uniformly at random, which suffers from slow wall-clock time for convergence due to high degrees of system heterogeneity and statistical heterogeneity. This paper aims to design an adaptive client sampling algorithm that tackles both system and statistical heterogeneity to minimize the wall-…
Tackling Privacy Heterogeneity in Differentially Private Federated Learning
Department of Information Engineering, The Chinese University of Hong Kong（信息工程系，香港中文大学）；School of Science and Engineering, Shenzhen Institute of Artificial Intelligence and Robotics for Society（科学与工程学院，深圳人工智能与机器人社会研究院）
Minimax optimization has seen a surge in interest with the advent of modern applications such as GANs, and it is inherently more challenging than simple minimization. The difficulty is exacerbated by the training data residing at multiple edge devices or \textit{clients}, especially when these clients can have heterogeneous datasets and local computation capabilities. We propose a general federated minimax optimization framework that subsumes such settings and several existing methods like Local SGDA. We show that naive aggregation of heterogeneous local progress results in optimizing a mismat…
AI Sweden（瑞典人工智能）；Beacom College（比科姆学院）；Dakota State University（达科他州立大学）
This paper proposes and investigates a new approach for detecting and preventing several different types of poisoning attacks from affecting a centralized Federated Learning model via average accuracy deviation detection (AADD). By comparing each client's accuracy to all clients' average accuracy, AADD detect clients with an accuracy deviation. The implementation is further able to blacklist clients that are considered poisoned, securing the global model from being affected by the poisoned nodes. The proposed implementation shows promising results in detecting poisoned clients and preventing t…
FedClust: Tackling Data Heterogeneity in Federated Learning through Weight-Driven Client Clustering
\ School of Computing；Informatics University of Louisiana at Lafayette 1 Th rv \"a ld Circle Lafayette, LA USA；\ School of Computer Science；Technology East China Normal University 1 Th rv \"a ld Circle Shanghai China；\ Department of Computer；Information Sciences University of Delaware 1 Th rv \"a ld Circle Newark, DE USA；School of Computing；Informatics University of Louisiana at Lafayette；School of Computer Science；Technology East China Normal University；Department of Computer；Information Sciences University of Delaware
FedSCAM (Federated Sharpness-Aware Minimization with Clustered Aggregation and Modulation): Scam-resistant SAM for Robust Federated Optimization in Heterogeneous Environments
Department of Computer Science, Lahore University of Management Sciences, Lahore, Pakistan（计算机科学系，拉合尔管理科学大学，拉合尔，巴基斯坦）
Mitigating Participation Imbalance Bias in Asynchronous Federated Learning
University of California, Riverside（加州大学河滨分校）；Indian Institute of Science（印度科学研究院）；University of Michigan, Ann Arbor（密歇根大学安娜堡分校）；DEVCOM Army Research Laboratory（陆军研究实验室）
FedCoSR: Personalized Federated Learning with Contrastive Shareable Representations for Label Heterogeneity in Non-IID Data
Department of Data Science and AI, Faculty of IT and Monash Energy Institute, Monash University（数据科学与人工智能系，IT学院和莫纳什能源研究所，莫纳什大学）；School of Computer Science and Technology, University of Electronic Science and Technology of China (UESTC), Chengdu, and Shenzhen Institute for Advanced Study of UESTC, Shenzhen, China（计算机科学与技术学院，电子科技大学（UESTC）成都校区，以及UESTC深圳高级研究所，深圳，中国）
Nowadays, the industrial Internet of Things (IIoT) has played an integral role in Industry 4.0 and produced massive amounts of data for industrial intelligence. These data locate on decentralized devices in modern factories. To protect the confidentiality of industrial data, federated learning (FL) was introduced to collaboratively train shared machine learning models. However, the local data collected by different devices skew in class distribution and degrade industrial FL performance. This challenge has been widely studied at the mobile edge, but they ignored the rapidly changing streaming …
The Hong Kong University of Science and Technology（香港科技大学）；The University of Hong Kong（香港大学）
The Federated Learning (FL) paradigm is known to face challenges under heterogeneous client data. Local training on non-iid distributed data results in deflected local optimum, which causes the client models drift further away from each other and degrades the aggregated global model's performance. A natural solution is to gather all client data onto the server, such that the server has a global view of the entire data distribution. Unfortunately, this reduces to regular training, which compromises clients' privacy and conflicts with the purpose of FL. In this paper, we put forth an idea to col…
We consider a standard federated learning (FL) architecture where a group of clients periodically coordinate with a central server to train a statistical model. We develop a general algorithmic framework called FedLin to tackle some of the key challenges intrinsic to FL, namely objective heterogeneity, systems heterogeneity, and infrequent and imprecise communication. Our framework is motivated by the observation that under these challenges, various existing FL algorithms suffer from a fundamental speed-accuracy conflict: they either guarantee linear convergence but to an incorrect point, or c…
King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
Federated learning (FL) is an emerging machine learning paradigm involving multiple clients, e.g., mobile phone devices, with an incentive to collaborate in solving a machine learning problem coordinated by a central server. FL was proposed in 2016 by Konečný et al. and McMahan et al. as a viable privacy-preserving alternative to traditional centralized machine learning since, by construction, the training data points are decentralized and never transferred by the clients to a central server. Therefore, to a certain degree, FL mitigates the privacy risks associated with centralized data collec…
Johns Hopkins University（约翰斯·霍普金斯大学）；Johns Hopkins University Applied Physics Laboratory（约翰斯·霍普金斯大学应用物理实验室）；Duke University（杜克大学）
Federated learning (FL) allows multiple clients to collaboratively train a deep learning model. One major challenge of FL is when data distribution is heterogeneous, i.e., differs from one client to another. Existing personalized FL algorithms are only applicable to narrow cases, e.g., one or two data classes per client, and therefore they do not satisfactorily address FL under varying levels of data heterogeneity. In this paper, we propose a novel framework, called DisTrans, to improve FL performance (i.e., model accuracy) via train and test-time distributional transformations along with a do…
Every Client Is an Environment: Federated De-confounding for Spatio-Temporal Forecasting
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Shanghai Jiao Tong University（上海交通大学）；Harbin Engineering University（哈尔滨工程大学）
Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction
Duke University（杜克大学）；University of Maryland, College Park（马里兰大学帕克分校）；George Mason University（乔治梅森大学）；Yale University（耶鲁大学）；The State University of New York at Buffalo（纽约州立大学布法罗分校）
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；Sony AI（索尼人工智能研究院）；Huawei Cloud & AI（华为云与计算）
The statistical heterogeneity of the non-independent and identically distributed (non-IID) data in local clients significantly limits the performance of federated learning. Previous attempts like FedProx, SCAFFOLD, MOON, FedNova and FedDyn resort to an optimization perspective, which requires an auxiliary term or re-weights local updates to calibrate the learning bias or the objective inconsistency. However, in addition to previous explorations for improvement in federated averaging, our analysis shows that another critical bottleneck is the poorer optima of client models in more heterogeneous…
University of Science and Technology of China（中国科学技术大学）
Federated learning (FL) allows multiple clients cooperatively train models without disclosing local data. However, the existing works fail to address all these practical concerns in FL: limited communication resources, dynamic network conditions and heterogeneous client properties, which slow down the convergence of FL. To tackle the above challenges, we propose a heterogeneity-aware FL framework, called FedCG, with adaptive client selection and gradient compression. Specifically, the parameter server (PS) selects a representative client subset considering statistical heterogeneity and sends t…
FedXDS: Leveraging Model Attribution Methods to counteract Data Heterogeneity in Federated Learning
Fraunhofer Heinrich Hertz Institute（弗劳恩霍夫海因里希·赫兹研究所）；Technical University of Berlin（柏林工业大学）；Berlin Institute for the Foundations of Learning and Data (BIFOLD)（柏林学习与数据基础研究所）
