Federated Learning (FL) represents a paradigm shift in the field of machine learning, offering an approach for a decentralized training of models across a multitude of devices while maintaining the privacy of local data. However, the dynamic nature of FL systems, characterized by the ongoing incorporation of new clients with potentially diverse data distributions and computational capabilities, poses a significant challenge to the stability and efficiency of these distributed learning networks. The seamless integration of new clients is imperative to sustain and enhance the performance and robustness of FL systems. This paper looks into the complexities of integrating new clients into existing FL systems and explores how data heterogeneity and varying data distribution (not independent and identically distributed) among them can affect model training, system efficiency, scalability and stability. Despite these challenges, the integration of new clients into FL systems presents opportunities to enhance data diversity, improve learning performance, and leverage distributed computational power. In contrast to other fields of application such as the distributed optimization of word predictions on Gboard (where federated learning once originated), there are usually only a few clients in the production environment, which is why information from each new client becomes all the more valuable. This paper outlines strategies for effective client selection strategies and solutions for ensuring system scalability and stability. Using the example of images from optical quality inspection, it offers insights into practical approaches. In conclusion, this paper proposes that addressing the challenges presented by new client integration is crucial to the advancement and efficiency of distributed learning networks, thus paving the way for the adoption of Federated Learning in production environments.
Dimensionality Reduction is a commonly used element in a machine learning pipeline that helps to extract important features from high-dimensional data. In this work, we explore an alternative federated learning system that enables integration of dimensionality reduced representations of distributed data prior to a supervised learning task, thus avoiding model sharing among the parties. We compare the performance of this approach on image classification tasks to three alternative frameworks: centralized machine learning, individual machine learning, and Federated Averaging, and analyze potentia…
Enhancing Clustered Federated Learning: Integration of Strategies and Improved Methodologies
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）理工学院）；Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)（深圳市人工智能与机器人研究院）；School of Engineering, Westlake University（西湖大学工学院）；Research Center for Industries of the Future, Westlake University（西湖大学未来产业研究中心）
LinkedIn Corporation（领英公司）；University of Michigan（密歇根大学）；RightScope Inc.（RightScope公司）
Cross-device federated learning (FL) has been well-studied from algorithmic, system scalability, and training speed perspectives. Nonetheless, moving from centralized training to cross-device FL for millions or billions of devices presents many risks, including performance loss, developer inertia, poor user experience, and unexpected application failures. In addition, the corresponding infrastructure, development costs, and return on investment are difficult to estimate. In this paper, we present a device-cloud collaborative FL platform that integrates with an existing machine learning platfor…
China Agricultural University（中国农业大学）；Beihang University（北京航空航天大学）；Lehigh University（理海大学）；University of Pennsylvania（宾夕法尼亚大学）
Along with the rapid expansion of information technology and digitalization of health data, there is an increasing concern on maintaining data privacy while garnering the benefits in medical field. Two critical challenges are identified: Firstly, medical data is naturally distributed across multiple local sites, making it difficult to collectively train machine learning models without data leakage. Secondly, in medical applications, data are often collected from different sources and views, resulting in heterogeneity and complexity that requires reconciliation. This paper aims to provide a gen…
DP2FL: Dual Prompt Personalized Federated Learning in Foundation Models
College of Software, Jilin University（吉林大学软件学院）；College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；College of Computer Science and Engineering, Changchun University of Technology（长春理工大学计算机科学与工程学院）
Towards Seamless Hierarchical Federated Learning under Intermittent Client Participation: A Stagewise Decision-Making Methodology
School of Informatics, Xiamen University（厦门大学信息学院）；Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University（同济大学上海智能自主系统研究所）；School of Electronic Science and Engineering, Xiamen University（厦门大学电子科学与技术学院）；Department of Electrical Engineering, University at Buffalo–SUNY（布法罗大学（SUNY）电气工程系）；Department of Electrical and Computer Engineering, Western University（温哥华大学电气与计算机工程系）；Department of Electrical and Computer Engineering, North Carolina State University（北卡罗来纳州立大学电气与计算机工程系）；iF-Labs, Beijing Teleinfo Technology Co., Ltd.（北京 Teleinfo 技术有限公司 iF 实验室）
In federated learning, models trained on local clients are distilled into a global model. Due to the permutation invariance arises in neural networks, it is necessary to match the hidden neurons first when executing federated learning with neural networks. Through the Bayesian nonparametric framework, Probabilistic Federated Neural Matching (PFNM) matches and fuses local neural networks so as to adapt to varying global model size and the heterogeneity of the data. In this paper, we propose a new method which extends the PFNM with a Kullback-Leibler (KL) divergence over neural components produc…
Beijing University of Posts and Telecommunications（北京邮电大学）
Implementing existing federated learning in massive Internet of Things (IoT) networks faces critical challenges such as imbalanced and statistically heterogeneous data and device diversity. To this end, we propose a semi-federated learning (SemiFL) framework to provide a potential solution for the realization of intelligent IoT. By seamlessly integrating the centralized and federated paradigms, our SemiFL framework shows high scalability in terms of the number of IoT devices even in the presence of computing-limited sensors. Furthermore, compared to traditional learning approaches, the propose…
Ericsson Research（爱立信研究院）；Ericsson Security Research（爱立信安全研究院）；KTH Royal Institute of Technology（瑞典皇家理工学院）
Machine Learning (ML) is an important enabler for optimizing, securing and managing mobile networks. This leads to increased collection and processing of data from network functions, which in turn may increase threats to sensitive end-user information. Consequently, mechanisms to reduce threats to end-user privacy are needed to take full advantage of ML. We seamlessly integrate Federated Learning (FL) into the 3GPP 5G Network Data Analytics (NWDA) architecture, and add a Multi-Party Computation (MPC) protocol for protecting the confidentiality of local updates. We evaluate the protocol and fin…
A Novel Collaborative Framework for Efficient Synchronization in Split Federated Learning over Wireless Networks
Department of Electrical and Computer Engineering, Concordia University（电气与计算机工程系，康科迪亚大学）；Department of Physics and Computer Science, Faculty of Science, Wilfrid Laurier University（物理与计算机科学系，科学学院，威尔弗里德·劳里埃大学）
This work addresses the problem of optimizing communications between server and clients in federated learning (FL). Current sampling approaches in FL are either biased, or non optimal in terms of server-clients communications and training stability. To overcome this issue, we introduce \textit{clustered sampling} for clients selection. We prove that clustered sampling leads to better clients representatitivity and to reduced variance of the clients stochastic aggregation weights in FL. Compatibly with our theory, we provide two different clustering approaches enabling clients aggregation based…
University of Virginia（弗吉尼亚大学）；University of Miami（迈阿密大学）；Sun Yat-Sen University（中山大学）
We advocate a new resource allocation framework, which we term resource rationing, for wireless federated learning (FL). Unlike existing resource allocation methods for FL, resource rationing focuses on balancing resources across learning rounds so that their collective impact on the federated learning performance is explicitly captured. This new framework can be integrated seamlessly with existing resource allocation schemes to optimize the convergence of FL. In particular, a novel "later-is-better" principle is at the front and center of resource rationing, which is validated empirically in …
CoM$^3$eT: A foundation model for medical image analysis through federated, multidimensional context integration
Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学MEVIS研究所）；RWTH Aachen University（亚琛工业大学）；Medizinische Hochschule Hannover（汉诺威医学院）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Charité – Universitätsmedizin Berlin（柏林夏里特医学院）；Freie Universität Berlin（柏林自由大学）；Humboldt-Universität zu Berlin（柏林洪堡大学）
Differential Privacy Personalized Federated Learning Based on Dynamically Sparsified Client Updates
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；No.208 Research Institute of China Ordnance Industries（中国兵器工业第二〇八研究所）
From Federated Learning to Quantum Federated Learning for Space-Air-Ground Integrated Networks
Hung Yen University of Technology and Education（兴安技术与教育大学）；University of Tasmania（塔斯马尼亚大学）；University of Alabama in Huntsville（阿拉巴马大学亨茨维尔分校）；Lac Hong University（乐鸿大学）；Manchester Metropolitan University（曼彻斯特城市大学）；International Hellenic University（国际希腊大学）
Simon Fraser University（西蒙菲莎大学）；Columbia University（哥伦比亚大学）
How can we debug a logistical regression model in a federated learning setting when seeing the model behave unexpectedly (e.g., the model rejects all high-income customers' loan applications)? The SQL-based training data debugging framework has proved effective to fix this kind of issue in a non-federated learning setting. Given an unexpected query result over model predictions, this framework automatically removes the label errors from training data such that the unexpected behavior disappears in the retrained model. In this paper, we enable this powerful framework for federated learning. The…
Federated Learning in Mobile Networks: A Comprehensive Case Study on Traffic Forecasting
Democritus University of Thrace（德谟克利特色雷斯大学）；Imperial College London（帝国理工学院）；Centre Tecnològic de Telecomunicacions de Catalunya (CTTC/CERCA)（加泰罗尼亚电信技术中心（CTTC/CERCA））；Nokia Bell Labs（诺基亚贝尔实验室）；IMT Atlantique（大西洋电信学校）；Inria（法国国家信息与自动化研究所）；LS2N（南特计算机科学与实验室（注：此处LS2N为实验室，按要求保留独立机构名））
Federated Unlearning Made Practical: Seamless Integration via Negated Pseudo-Gradients
Department of Computer Science and Engineering, University of Bologna（计算机科学与工程系，博洛尼亚大学）；Department of Information and Electrical Engineering and Applied Mathematics, University of Salerno（信息与电气工程及应用数学系，萨勒诺大学）
Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）
Conventional machine learning techniques are conducted in a centralized manner. Recently, the massive volume of generated wireless data, the privacy concerns and the increasing computing capabilities of wireless end-devices have led to the emergence of a promising decentralized solution, termed as Wireless Federated Learning (WFL). In this first of the two parts paper, we present the application of WFL in the sixth generation of wireless networks (6G), which is envisioned to be an integrated communication and computing platform. After analyzing the key concepts of WFL, we discuss the core chal…
A Distributed Computation Model Based on Federated Learning Integrates Heterogeneous models and Consortium Blockchain for Solving Time-Varying Problems
University of Macau（澳门大学）；Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人研究院）；Northwestern Polytechnical University（西北工业大学）
ConsenSys Health；Imperial College London（帝国理工学院）；UC San Diego（加州大学圣地亚哥分校）；Johns Hopkins Berman Institute of Bioethics（约翰斯·霍普金斯大学伯曼生物伦理研究所）
We propose a novel architecture for federated learning within healthcare consortia. At the heart of the solution is a unique integration of privacy preserving technologies, built upon native enterprise blockchain components available in the Ethereum ecosystem. We show how the specific characteristics and challenges of healthcare consortia informed our design choices, notably the conception of a new Secure Aggregation protocol assembled with a protected hardware component and an encryption toolkit native to Ethereum. Our architecture also brings in a privacy preserving audit trail that logs eve…
