An Efficiency-boosting Client Selection Scheme for Federated Learning with Fairness Guarantee
South China University of Technology（华南理工大学）；the University of Warwick（华威大学）；State University of New York, New Paltz（纽约州立大学新帕尔兹分校）；The University of Sydney（悉尼大学）
Tiansheng Huang, Weiwei Lin, Wentai Wu, Ligang He, Keqin Li, Albert Y. Zomaya
备注：
Accepted by IEEE TPDS. DOI: 10.1109/TPDS.2020.3040887
The issue of potential privacy leakage during centralized AI's model training has drawn intensive concern from the public. A Parallel and Distributed Computing (or PDC) scheme, termed Federated Learning (FL), has emerged as a new paradigm to cope with the privacy issue by allowing clients to perform model training locally, without the necessity to upload their personal sensitive data. In FL, the number of clients could be sufficiently large, but the bandwidth available for model distribution and re-upload is quite limited, making it sensible to only involve part of the volunteers to participate in the training process. The client selection policy is critical to an FL process in terms of training efficiency, the final model's quality as well as fairness. In this paper, we will model the fairness guaranteed client selection as a Lyapunov optimization problem and then a C2MAB-based method is proposed for estimation of the model exchange time between each client and the server, based on which we design a fairness guaranteed algorithm termed RBCS-F for problem-solving. The regret of RBCS-F is strictly bounded by a finite constant, justifying its theoretical feasibility. Barring the theoretical results, more empirical data can be derived from our real training experiments on public datasets.
Fairness-Aware Client Selection for Federated Learning
Nanyang Technological University (NTU)（南洋理工大学）；Alibaba-NTU Singapore Joint Research Institute（阿里巴巴-新加坡南洋理工大学联合研究院）；Alibaba Group（阿里巴巴集团）；University of Science and Technology of China（中国科学技术大学）
Emulating Full Participation: An Effective and Fair Client Selection Strategy for Federated Learning
The Chinese University of Hong Kong（香港中文大学）；Zhejiang Lab（之江实验室）；School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区网络空间安全学院）；Tsinghua University（清华大学）
FairGFL: Privacy-Preserving Fairness-Aware Federated Learning with Overlapping Subgraphs
National Engineering Laboratory for Big Data Analytics (NEL-BDA)（大数据分析国家工程实验室）；School of Mathematics and Statistics（数学与统计学学院）；Ministry of Education Key Lab for Intelligent Networks and Network Security（教育部智能网络与网络安全重点实验室）；School of Computer Science and Technology（计算机科学与技术学院）
Federated learning (FL) is a privacy-preserving learning technique that enables distributed computing devices to train shared learning models across data silos collaboratively. Existing FL works mostly focus on designing advanced FL algorithms to improve the model performance. However, the economic considerations of the clients, such as fairness and incentive, are yet to be fully explored. Without such considerations, self-motivated clients may lose interest and leave the federation. To address this problem, we designed a novel incentive mechanism that involves a client selection process to re…
Federated learning (FL) has emerged as an important machine learning paradigm where a global model is trained based on the private data from distributed clients. However, most of existing FL algorithms cannot guarantee the performance fairness towards different groups because of data distribution shift over groups. In this paper, we formulate the problem of unified group fairness on FL, where the groups can be formed by clients (including existing clients and newly added clients) and sensitive attribute(s). To solve this problem, we first propose a general fair federated framework. Then we con…
The Chinese University of Hong Kong（香港中文大学）；NVIDIA（英伟达）
How to ensure fairness is an important topic in federated learning (FL). Recent studies have investigated how to reward clients based on their contribution (collaboration fairness), and how to achieve uniformity of performance across clients (performance fairness). Despite achieving progress on either one, we argue that it is critical to consider them together, in order to engage and motivate more diverse clients joining FL to derive a high-quality global model. In this work, we propose a novel method to optimize both types of fairness simultaneously. Specifically, we propose to estimate clien…
Federated Learning (FL) enables data owners to train a shared global model without sharing their private data. Unfortunately, FL is susceptible to an intrinsic fairness issue: due to heterogeneity in clients' data distributions, the final trained model can give disproportionate advantages across the participating clients. In this work, we present Equal and Equitable Federated Learning (E2FL) to produce fair federated learning models by preserving two main fairness properties, equity and equality, concurrently. We validate the efficiency and fairness of E2FL in different real-world FL applicati…
Enhancing One-Shot Federated Learning Through Data and Ensemble Co-Boosting
University of Science and Technology of China（中国科学技术大学）；Hong Kong Baptist University（香港浸会大学）；University of Maryland College Park（马里兰大学学院公园分校）；The University of Sydney（悉尼大学）
Nanyang Technological University（南洋理工大学）；Alibaba-NTU Singapore Joint Research Institute（阿里巴巴-南洋理工大学新加坡联合研究院）；Alibaba Group（阿里巴巴集团）；The University of British Columbia（不列颠哥伦比亚大学）
Fairness-Aware Job Scheduling for Multi-Job Federated Learning
Nanyang Technological University（南洋理工大学）；DSO National Laboratories（DSO国家实验室）；Alibaba Group（阿里巴巴集团）；Alibaba-NTU Singapore Joint Research Institute（阿里巴巴-新加坡南洋理工大学联合研究院）
Federated learning (FL) has gain growing interests for its capability of learning from distributed data sources collectively without the need of accessing the raw data samples across different sources. So far FL research has mostly focused on improving the performance, how the algorithmic disparity will be impacted for the model learned from FL and the impact of algorithmic disparity on the utility inconsistency are largely unexplored. In this paper, we propose an FL framework to jointly consider performance consistency and algorithmic fairness across different local clients (data sources). We…
In the federated learning setting, multiple clients jointly train a model under the coordination of the central server, while the training data is kept on the client to ensure privacy. Normally, inconsistent distribution of data across different devices in a federated network and limited communication bandwidth between end devices impose both statistical heterogeneity and expensive communication as major challenges for federated learning. This paper proposes an algorithm to achieve more fairness and accuracy in federated learning (FedFa). It introduces an optimization scheme that employs a dou…
In Federated Learning (FL), the clients learn a single global model (FedAvg) through a central aggregator. In this setting, the non-IID distribution of the data across clients restricts the global FL model from delivering good performance on the local data of each client. Personalized FL aims to address this problem by finding a personalized model for each client. Recent works widely report the average personalized model accuracy on a particular data split of a dataset to evaluate the effectiveness of their methods. However, considering the multitude of personalization approaches proposed, it …
Mila – Quebec AI Institute（Mila – 魁北克人工智能研究所）；HEC Montréal（蒙特利尔高等商学院）；University of Washington Tacoma（华盛顿大学塔科马分校）；Ghent University（根特大学）
Group fairness ensures that the outcome of machine learning (ML) based decision making systems are not biased towards a certain group of people defined by a sensitive attribute such as gender or ethnicity. Achieving group fairness in Federated Learning (FL) is challenging because mitigating bias inherently requires using the sensitive attribute values of all clients, while FL is aimed precisely at protecting privacy by not giving access to the clients' data. As we show in this paper, this conflict between fairness and privacy in FL can be resolved by combining FL with Secure Multiparty Computa…
Towards Collaborative Fairness in Federated Learning Under Imbalanced Covariate Shift
The Pennsylvania State University（宾夕法尼亚州立大学）；State University of New York at Albany（纽约州立大学阿尔巴尼分校）；University of Minnesota Twin Cities（明尼苏达大学双城分校）；Dalian University of Technology（大连理工大学）
Fairness-Aware Federated Learning with Trajectory Shapley Value
Faculty of Mathematics, Ecole Normale Supérieure Paris-Saclay（巴黎-萨克雷大学数学系）；Chair for Dynamics, Control, Machine Learning and Numerics – Alexander von Humboldt Professorship, Department of Mathematics, Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔兰根-纽伦堡弗里德里希-亚历山大大学数学系）
A key factor in developing high performing machine learning models is the availability of sufficiently large datasets. This work is motivated by applications arising in Software as a Service (SaaS) companies where there exist numerous similar yet disjoint datasets from multiple client companies. To overcome the challenges of insufficient data without explicitly aggregating the clients' datasets due to privacy concerns, one solution is to collect more data for each individual client, another is to privately aggregate information from models trained on each client's data. In this work, two appro…
Indian Institute of Technology, Bhilai（印度理工学院比莱分校）
The traditional framework of federated learning (FL) requires each client to re-train their models in every iteration, making it infeasible for resource-constrained mobile devices to train deep-learning (DL) models. Split learning (SL) provides an alternative by using a centralized server to offload the computation of activations and gradients for a subset of the model but suffers from problems of slow convergence and lower accuracy. In this paper, we implement PFSL, a new framework of distributed split learning where a large number of thin clients perform transfer learning in parallel, starti…
FUSE: Fairness-aware Update Screening against Byzantine Exploitation in $q$-Fair Federated Learning
George Mason University（乔治梅森大学）；The University of Hong Kong（香港大学）；Huazhong University of Science and Technology（华中科技大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；The Hong Kong Polytechnic University（香港理工大学）
Fair Federated Medical Image Classification Against Quality Shift via Inter-Client Progressive State Matching
School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）；Lassonde School of Engineering, York University（约克大学拉萨onde工程学院）
University of Pennsylvania（宾夕法尼亚大学）；Google Research（谷歌研究院）；Yale（耶鲁大学）
In this paper, we initiate a study of functional minimization in Federated Learning. First, in the semi-heterogeneous setting, when the marginal distributions of the feature vectors on client machines are identical, we develop the federated functional gradient boosting (FFGB) method that provably converges to the global minimum. Subsequently, we extend our results to the fully-heterogeneous setting (where marginal distributions of feature vectors may differ) by designing an efficient variant of FFGB called FFGB.C, with provable convergence to a neighborhood of the global minimum within a radiu…
FAIRVAR: Fair Federated Learning via Variance Regularization
Department of Computing Science, Umeå University, Umeå, Sweden（计算机科学系，乌梅大学，乌梅，瑞典）；Department of Mathematics and Mathematical Statistics, Umeå University, Umeå, Sweden（数学与统计学系，乌梅大学，乌梅，瑞典）
Federated learning (FL) is a distributed learning technique that trains a shared model over distributed data in a privacy-preserving manner. Unfortunately, FL's performance degrades when there is (i) variability in client characteristics in terms of computational and memory resources (system heterogeneity) and (ii) non-IID data distribution across clients (statistical heterogeneity). For example, slow clients get dropped in FL schemes, such as Federated Averaging (FedAvg), which not only limits overall learning but also biases results towards fast clients. We propose FedPrune; a system that ta…
Submodular Maximization Approaches for Equitable Client Selection in Federated Learning
Elmore Family School of Electrical and Computer Engineering at Purdue University（普渡大学埃尔莫尔家族电气与计算机工程学院）；School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）
