Federated learning (FL) is a distributed machine learning approach involving multiple clients collaboratively training a shared model. Such a system has the advantage of more training data from multiple clients, but data can be non-identically and independently distributed (non-i.i.d.). Privacy and integrity preserving features such as differential privacy (DP) and robust aggregation (RA) are commonly used in FL. In this work, we show that on common deep learning tasks, the performance of FL models differs amongst clients and situations, and FL models can sometimes perform worse than local models due to non-i.i.d. data. Secondly, we show that incorporating DP and RA degrades performance further. Then, we conduct an ablation study on the performance impact of different combinations of common personalization approaches for FL, such as finetuning, mixture-of-experts ensemble, multi-task learning, and knowledge distillation. It is observed that certain combinations of personalization approaches are more impactful in certain scenarios while others always improve performance, and combination approaches are better than individual ones. Most clients obtained better performance with combined personalized FL and recover from performance degradation caused by non-i.i.d. data, DP, and RA.
Most personalised federated learning (FL) approaches assume that raw data of all clients are defined in a common subspace i.e. all clients store their data according to the same schema. For real-world applications, this assumption is restrictive as clients, having their own systems to collect and then store data, may use heterogeneous data representations. We aim at filling this gap. To this end, we propose a general framework coined FLIC that maps client's data onto a common feature space via local embedding functions. The common feature space is learnt in a federated manner using Wasserstein…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）；Criteo AI Lab（Criteo人工智能实验室）；ENS Paris-Saclay（巴黎萨克雷高等师范学院）；Ecole Polytechnique（巴黎综合理工学院）
Personalised federated learning (FL) aims at collaboratively learning a machine learning model taylored for each client. Albeit promising advances have been made in this direction, most of existing approaches works do not allow for uncertainty quantification which is crucial in many applications. In addition, personalisation in the cross-device setting still involves important issues, especially for new clients or those having small number of observations. This paper aims at filling these gaps. To this end, we propose a novel methodology coined FedPop by recasting personalised FL into the popu…
Federated Learning (FL) is an emerging approach for collaboratively training Deep Neural Networks (DNNs) on mobile devices, without private user data leaving the devices. Previous works have shown that non-Independent and Identically Distributed (non-IID) user data harms the convergence speed of the FL algorithms. Furthermore, most existing work on FL measures global-model accuracy, but in many cases, such as user content-recommendation, improving individual User model Accuracy (UA) is the real objective. To address these issues, we propose a Multi-Task FL (MTFL) algorithm that introduces non-…
Robust Federated Personalised Mean Estimation for the Gaussian Mixture Model
School of Technology \& Comp. Sc. Tata Institute of Fundamental Research Mumbai, India；Department of Electrical Engineering University of California Los Angeles, USA
Federated Learning (FL) has evolved as a promising technique to handle distributed machine learning across edge devices. A single neural network (NN) that optimises a global objective is generally learned in most work in FL, which could be suboptimal for edge devices. Although works finding a NN personalised for edge device specific tasks exist, they lack generalisation and/or convergence guarantees. In this paper, a novel communication efficient FL algorithm for personalised learning in a wireless setting with guarantees is presented. The algorithm relies on finding a ``better`` empirical est…
PTOPOFL: Topology-Guided Personalised Federated Learning with Persistent-Homology Client Descriptors
Université Sorbonne Paris Nord（索邦巴黎北大学）；LAGA（拉加数学研究所）；CNRS（国家科学研究中心）；UMR 7539（7539号研究单位）；Laboratoire d’excellence Infibrex（卓越实验室Infibrex）；Instituto de Hortofruticultura Subtropical y Mediterránea La Mayora (IHSM)（亚得里亚地中海亚热带园艺研究所（IHSM））；Universidad de Málaga–Consejo Superior de Investigaciones Científicas（马拉加大学–高级科学研究理事会）
With growing concerns regarding data privacy and rapid increase in data volume, Federated Learning(FL) has become an important learning paradigm. However, jointly learning a deep neural network model in a FL setting proves to be a non-trivial task because of the complexities associated with the neural networks, such as varied architectures across clients, permutation invariance of the neurons, and presence of non-linear transformations in each layer. This work introduces a novel Federated Heterogeneous Neural Networks (FedHeNN) framework that allows each client to build a personalised model wi…
University of Bristol（布里斯托大学）；Toshiba Europe Ltd.（东芝欧洲有限公司）
Data-enabled cities are recently accelerated and enhanced with automated learning for improved Smart Cities applications. In the context of an Internet of Things (IoT) ecosystem, the data communication is frequently costly, inefficient, not scalable and lacks security. Federated Learning (FL) plays a pivotal role in providing privacy-preserving and communication efficient Machine Learning (ML) frameworks. In this paper we evaluate the feasibility of FL in the context of a Smart Cities Street Light Monitoring application. FL is evaluated against benchmarks of centralised and (fully) personalise…
FedFusion: Federated Learning with Diversity- and Cluster-Aware Encoders for Robust Adaptation under Label Scarcity
Department of Computer Science University of the Western Cape（计算机科学系乌得勒支大学）；Department of Computer Science Missouri University of Science and Technology（计算机科学系密苏里科技大学）
Embedding-Based Federated Learning with Runtime Governance for Iron Deficiency Prediction
Theoretical Physics University of Cambridge Cambridge, UK；Translational AI Laboratory, Dept. of Laboratory Medicine Amsterdam UMC Amsterdam, The Netherlands；Precision Health University Research Institute Queen Mary Univ. of London London, UK；Department of Medicine University of Cambridge Cambridge Biomedical Campus Cambridge, UK；Transplant Cambridge Biomedical Campus Cambridge, UK；Dept. of Cardiology Amsterdam Cardiovascular Sciences Amsterdam UMC Amsterdam, The Netherlands
The heterogeneity of hardware and data is a well-known and studied problem in the community of Federated Learning (FL) as running under heterogeneous settings. Recently, custom-size client models trained with Knowledge Distillation (KD) has emerged as a viable strategy for tackling the heterogeneity challenge. However, previous efforts in this direction are aimed at client model tuning rather than their impact onto the knowledge aggregation of the global model. Despite performance of global models being the primary objective of FL systems, under heterogeneous settings client models have receiv…
Blockchain-Enabled Privacy-Preserving Second-Order Federated Edge Learning in Personalized Healthcare
Shanghai Key Laboratory of Intelligent Information Processing Lab, Fudan University, China（上海智能信息处理实验室，复旦大学，中国）；Turku Intelligent Embedded and Robotic Systems (TIERS) Lab, University of Turku, Finland（图尔库智能嵌入式与机器人系统（TIERS）实验室，图尔库大学，芬兰）；TIERS Lab, University of Turku, Finland（TIERS实验室，图尔库大学，芬兰）；Department of Electrical Engineering, Islamic University of Madinah, Saudi Arabia（电气工程系，麦加伊斯兰大学，沙特阿拉伯）；School of Information Science and Technology, Fudan University, Shanghai, China（信息科学与技术学院，复旦大学，上海，中国）；Faculty of Technology, University of Turku, Finland（技术学院，图尔库大学，芬兰）
