Fedcompass: Federated Clustered and Periodic Aggregation Framework for Hybrid Classical-Quantum Models
1 School of Artificial Intelligence；Computer Science, Jiangnan University 2 Antai College of Economics \& Management, Shanghai Jiao Tong University 3 School of Computer Science, Shanghai Jiao Tong University 4 Cloud Computing；Distributed Systems Laboratory, University of Melbourne
Federated learning enables collaborative model training across decentralized clients under privacy constraints. Quantum computing offers potential for alleviating computational and communication burdens in federated learning, yet hybrid classical-quantum federated learning remains susceptible to performance degradation under non-IID data. To address this,we propose FEDCOMPASS, a layered aggregation framework for hybrid classical-quantum federated learning. FEDCOMPASS employs spectral clustering to group clients by class distribution similarity and performs cluster-wise aggregation for classical feature extractors. For quantum parameters, it uses circular mean aggregation combined with adaptive optimization to ensure stable global updates. Experiments on three benchmark datasets show that FEDCOMPASS improves test accuracy by up to 10.22% and enhances convergence stability under non-IID settings, outperforming six strong federated learning baselines.
FedCompass: Efficient Cross-Silo Federated Learning on Heterogeneous Client Devices using a Computing Power Aware Scheduler
Argonne National Laboratory（阿贡国家实验室）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；VMWare Research（威睿研究院）；The University of Chicago（芝加哥大学）
Traditionally, clustered federated learning groups clients with the same data distribution into a cluster, so that every client is uniquely associated with one data distribution and helps train a model for this distribution. We relax this hard association assumption to soft clustered federated learning, which allows every local dataset to follow a mixture of multiple source distributions. We propose FedSoft, which trains both locally personalized models and high-quality cluster models in this setting. FedSoft limits client workload by using proximal updates to require the completion of only on…
Australian Artificial Intelligence Institute, University of Technology Sydney（悉尼科技大学澳大利亚人工智能研究所）；University of Washington, Seattle（西雅图华盛顿大学）；University of Maryland, College Park（马里兰大学帕克分校）
Knowledge sharing and model personalization are essential components to tackle the non-IID challenge in federated learning (FL). Most existing FL methods focus on two extremes: 1) to learn a shared model to serve all clients with non-IID data, and 2) to learn personalized models for each client, namely personalized FL. There is a trade-off solution, namely clustered FL or cluster-wise personalized FL, which aims to cluster similar clients into one cluster, and then learn a shared model for all clients within a cluster. This paper is to revisit the research of clustered FL by formulating them i…
Personalized federated learning (FL) aims to train model(s) that can perform well for individual clients that are highly data and system heterogeneous. Most work in personalized FL, however, assumes using the same model architecture at all clients and increases the communication cost by sending/receiving models. This may not be feasible for realistic scenarios of FL. In practice, clients have highly heterogeneous system-capabilities and limited communication resources. In our work, we propose a personalized FL framework, PerFed-CKT, where clients can use heterogeneous model architectures and d…
Single-Round Clustered Federated Learning via Data Collaboration Analysis for Non-IID Data
Graduate School of Science and Technology, University of Tsukuba（茨川大学理学技术研究生院）；Center for Artificial Intelligence Research, Tsukuba Institute for Advanced Research, University of Tsukuba（茨川大学先进研究所人工智能研究中心）；Institute of Systems and Information Engineering, University of Tsukuba（茨川大学系统与信息工程研究所）
Enhancing Clustered Federated Learning: Integration of Strategies and Improved Methodologies
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）理工学院）；Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)（深圳市人工智能与机器人研究院）；School of Engineering, Westlake University（西湖大学工学院）；Research Center for Industries of the Future, Westlake University（西湖大学未来产业研究中心）
Texas A&M University（德克萨斯农工大学）；Texas A&M University at Qatar（德克萨斯农工大学卡塔尔分校）；Qatar Foundation Research, Development and Innovation RDI（卡塔尔基金会研究、开发与创新（RDI））；Tennessee Tech University（田纳西理工大学）
This paper proposes using communication pipelining to enhance the wireless spectrum utilization efficiency and convergence speed of federated learning in mobile edge computing applications. Due to limited wireless sub-channels, a subset of the total clients is scheduled in each iteration of federated learning algorithms. On the other hand, the scheduled clients wait for the slowest client to finish its computation. We propose to first cluster the clients based on the time they need per iteration to compute the local gradients of the federated learning model. Then, we schedule a mixture of clie…
Distilling A Universal Expert from Clustered Federated Learning
Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education（教育部符号计算与知识工程重点实验室）；College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；Australian Artificial Intelligence Institute, FEIT, University of Technology Sydney（悉尼大学技术学院澳大利亚人工智能研究所）；College of Computer Science, Inner Mongolia University（内蒙古大学计算机科学学院）
Novel clustered federated learning based on local loss
LMIB, School of Mathematical Sciences, Beihang University（力学系，数学科学学院，北航）；College of Science, China Agricultural University（中国农业大学，理学院）；School of Mathematical Sciences, Nanjing Normal University（南京师范大学，数学科学学院）；School of Mathematical Sciences, Beihang University（力学系，数学科学学院，北航）
A Joint Gradient and Loss Based Clustered Federated Learning Design
University of Miami（迈阿密大学）；Frost Institute for Data Science and Computing, University of Miami（迈阿密大学弗罗斯特数据科学与计算研究所）；Zhejiang University（浙江大学）；North Carolina State University（北卡罗来纳州立大学）
Institute of Communication Acoustics, Ruhr-Universit\"at Bochum 44780 Bochum, Germany
In this paper we introduce a realistic and challenging, multi-source and multi-room acoustic environment and an improved algorithm for the estimation of source-dominated microphone clusters in acoustic sensor networks. Our proposed clustering method is based on a single microphone per node and on unsupervised clustered federated learning which employs a light-weight autoencoder model. We present an improved clustering control strategy that takes into account the variability of the acoustic scene and allows the estimation of a dynamic range of clusters using reduced amounts of training data. Th…
This work addresses the problem of optimizing communications between server and clients in federated learning (FL). Current sampling approaches in FL are either biased, or non optimal in terms of server-clients communications and training stability. To overcome this issue, we introduce \textit{clustered sampling} for clients selection. We prove that clustered sampling leads to better clients representatitivity and to reduced variance of the clients stochastic aggregation weights in FL. Compatibly with our theory, we provide two different clustering approaches enabling clients aggregation based…
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Guangzhou HKUST Fok Ying Tung Research Institute（广州香港科技大学霍英东研究院）；Baidu Research, Baidu Inc.（百度研究院，百度公司）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；BOSS Zhipin（BOSS直聘）
University of Padova（帕多瓦大学）；Politecnico di Torino（都灵理工大学）
Federated Learning (FL) has recently emerged as a possible way to tackle the domain shift in real-world Semantic Segmentation (SS) without compromising the private nature of the collected data. However, most of the existing works on FL unrealistically assume labeled data in the remote clients. Here we propose a novel task (FFREEDA) in which the clients' data is unlabeled and the server accesses a source labeled dataset for pre-training only. To solve FFREEDA, we propose LADD, which leverages the knowledge of the pre-trained model by employing self-supervision with ad-hoc regularization techniq…
Shenzhen University（深圳大学）；Macquarie University（麦考瑞大学）；Georgia State University（佐治亚州立大学）
Federated Learning (FL) is an emerging decentralized learning framework through which multiple clients can collaboratively train a learning model. However, a major obstacle that impedes the wide deployment of FL lies in massive communication traffic. To train high dimensional machine learning models (such as CNN models), heavy communication traffic can be incurred by exchanging model updates via the Internet between clients and the parameter server (PS), implying that the network resource can be easily exhausted. Compressing model updates is an effective way to reduce the traffic amount. Howev…
We address the problem of federated learning (FL) where users are distributed and partitioned into clusters. This setup captures settings where different groups of users have their own objectives (learning tasks) but by aggregating their data with others in the same cluster (same learning task), they can leverage the strength in numbers in order to perform more efficient federated learning. For this new framework of clustered federated learning, we propose the Iterative Federated Clustering Algorithm (IFCA), which alternately estimates the cluster identities of the users and optimizes model pa…
FedSCAM (Federated Sharpness-Aware Minimization with Clustered Aggregation and Modulation): Scam-resistant SAM for Robust Federated Optimization in Heterogeneous Environments
Department of Computer Science, Lahore University of Management Sciences, Lahore, Pakistan（计算机科学系，拉合尔管理科学大学，拉合尔，巴基斯坦）
Classical federated learning approaches yield significant performance degradation in the presence of Non-IID data distributions of participants. When the distribution of each local dataset is highly different from the global one, the local objective of each client will be inconsistent with the global optima which incur a drift in the local updates. This phenomenon highly impacts the performance of clients. This is while the primary incentive for clients to participate in federated learning is to obtain better personalized models. To address the above-mentioned issue, we present a new algorithm…
Fraunhofer Heinrich Hertz Institute（弗劳恩霍夫海因里希·赫兹研究所）；Technische Universität Berlin（柏林工业大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Korea University（高丽大学）
Federated Learning (FL) is currently the most widely adopted framework for collaborative training of (deep) machine learning models under privacy constraints. Albeit it's popularity, it has been observed that Federated Learning yields suboptimal results if the local clients' data distributions diverge. To address this issue, we present Clustered Federated Learning (CFL), a novel Federated Multi-Task Learning (FMTL) framework, which exploits geometric properties of the FL loss surface, to group the client population into clusters with jointly trainable data distributions. In contrast to existin…
To reduce negative environmental impacts, power stations and energy grids need to optimize the resources required for power production. Thus, predicting the energy consumption of clients is becoming an important part of every energy management system. Energy usage information collected by the clients' smart homes can be used to train a deep neural network to predict the future energy demand. Collecting data from a large number of distributed clients for centralized model training is expensive in terms of communication resources. To take advantage of distributed data in edge systems, centralize…
Southeast University（东南大学）；Harbin Institute of Technology（哈尔滨工业大学）；University of Oulu（奥卢大学）
In this paper, the content popularity prediction problem in fog radio access networks (F-RANs) is investigated. Based on clustered federated learning, we propose a novel mobility-aware popularity prediction policy, which integrates content popularities in terms of local users and mobile users. For local users, the content popularity is predicted by learning the hidden representations of local users and contents. Initial features of local users and contents are generated by incorporating neighbor information with self information. Then, dual-channel neural network (DCNN) model is introduced to …
UC San Diego（加州大学圣地亚哥分校）；Czech Technical University（捷克技术大学）；UCF（中佛罗里达大学）
Clustered federated learning (FL) has been shown to produce promising results by grouping clients into clusters. This is especially effective in scenarios where separate groups of clients have significant differences in the distributions of their local data. Existing clustered FL algorithms are essentially trying to group together clients with similar distributions so that clients in the same cluster can leverage each other's data to better perform federated learning. However, prior clustered FL algorithms attempt to learn these distribution similarities indirectly during training, which can b…
Clustered Federated Multitask Learning (CFL) was introduced as an efficient scheme to obtain reliable specialized models when data is imbalanced and distributed in a non-i.i.d. (non-independent and identically distributed) fashion amongst clients. While a similarity measure metric, like the cosine similarity, can be used to endow groups of the client with a specialized model, this process can be arduous as the server should involve all clients in each of the federated learning rounds. Therefore, it is imperative that a subset of clients is selected periodically due to the limited bandwidth and…
School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；Cyberspace Security Research Center, Peng Cheng Laboratory（鹏城实验室网络空间安全研究中心）；School of Electrical and Data Engineering, University of Technology Sydney（悉尼科技大学电气与数据工程学院）；Department of Computer Science and Technology, Fuzhou University（福州大学计算机科学与技术学院）；Department of Computer Science, Electrical and Space Engineering, Lulea University of Technology（吕勒奥理工大学计算机、电气与空间工程系）
In recent years, mobile clients' computing ability and storage capacity have greatly improved, efficiently dealing with some applications locally. Federated learning is a promising distributed machine learning solution that uses local computing and local data to train the Artificial Intelligence (AI) model. Combining local computing and federated learning can train a powerful AI model under the premise of ensuring local data privacy while making full use of mobile clients' resources. However, the heterogeneity of local data, that is, Non-independent and identical distribution (Non-IID) and imb…
DAG-ACFL: Asynchronous Clustered Federated Learning based on DAG-DLT
Information Countermeasure Technique Institute（信息对抗技术研究所）；School of Cyberspace Science（网络空间科学学院）；Faculty of Computing（计算学部）；Harbin Institute of Technology（哈尔滨工业大学）
Flexible Clustered Federated Learning for Client-Level Data Distribution Shift
Chongqing University（重庆大学）；College of Computer Science, Chongqing University（重庆大学计算机学院）；School of Microelectronics and Communication Engineering, Chongqing University（重庆大学微电子与通信工程学院）
