Federated learning (FL) is a collaborative and privacy-preserving Machine Learning paradigm, allowing the development of robust models without the need to centralize sensitive data. A critical challenge in FL lies in fairly and accurately allocating contributions from diverse participants. Inaccurate allocation can undermine trust, lead to unfair compensation, and thus participants may lack the incentive to join or actively contribute to the federation. Various remuneration strategies have been proposed to date, including auction-based approaches and Shapley-value-based methods, the latter offering a means to quantify the contribution of each participant. However, little to no work has studied the stability of these contribution evaluation methods. In this paper, we evaluate participant contributions in federated learning using gradient-based model reconstruction techniques with Shapley values and compare the round-based contributions to a classic data contribution measurement scheme. We provide an extensive analysis of the discrepancies of Shapley values across a set of aggregation strategies and examine them on an overall and a per-client level. We show that, between different aggregation techniques, Shapley values lead to unstable reward allocations among participants. Our analysis spans various data heterogeneity distributions, including independent and identically distributed (IID) and non-IID scenarios.
Shapley Value is commonly adopted to measure and incentivize client participation in federated learning. In this paper, we show -- theoretically and through simulations -- that Shapley Value underestimates the contribution of a common type of client: the Maverick. Mavericks are clients that differ both in data distribution and data quantity and can be the sole owners of certain types of data. Selecting the right clients at the right moment is important for federated learning to reduce convergence times and improve accuracy. We propose FedEMD, an adaptive client selection strategy based on the …
Federated Learning is introduced to protect privacy by distributing training data into multiple parties. Each party trains its own model and a meta-model is constructed from the sub models. In this way the details of the data are not disclosed in between each party. In this paper we investigate the model interpretation methods for Federated Learning, specifically on the measurement of feature importance of vertical Federated Learning where feature space of the data is divided into two parties, namely host and guest. For host party to interpret a single prediction of vertical Federated Learning…
University of British Columbia（不列颠哥伦比亚大学）；Huawei Technologies Canada Co.（华为技术加拿大公司）；Simon Fraser University（西蒙弗雷泽大学）；KTH Royal Institute of Technology（KTH皇家理工学院）
Federated learning is an emerging decentralized machine learning scheme that allows multiple data owners to work collaboratively while ensuring data privacy. The success of federated learning depends largely on the participation of data owners. To sustain and encourage data owners' participation, it is crucial to fairly evaluate the quality of the data provided by the data owners and reward them correspondingly. Federated Shapley value, recently proposed by Wang et al. [Federated Learning, 2020], is a measure for data value under the framework of federated learning that satisfies many desired …
Fairness-Aware Federated Learning with Trajectory Shapley Value
Faculty of Mathematics, Ecole Normale Supérieure Paris-Saclay（巴黎-萨克雷大学数学系）；Chair for Dynamics, Control, Machine Learning and Numerics – Alexander von Humboldt Professorship, Department of Mathematics, Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔兰根-纽伦堡弗里德里希-亚历山大大学数学系）
Federated Learning for Data Market: Shapley-UCB for Seller Selection and Incentives
School of Intelligent Systems Engineering, Sun Yat-Sen University（中山大学智能系统工程学院）；Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算机系）
Massachusetts General Hospital（麻省总医院）；Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）
Federated Learning (FL) wherein multiple institutions collaboratively train a machine learning model without sharing data is becoming popular. Participating institutions might not contribute equally, some contribute more data, some better quality data or some more diverse data. To fairly rank the contribution of different institutions, Shapley value (SV) has emerged as the method of choice. Exact SV computation is impossibly expensive, especially when there are hundreds of contributors. Existing SV computation techniques use approximations. However, in healthcare where the number of contributi…
The paradigm of Federated learning (FL) deals with multiple clients participating in collaborative training of a machine learning model under the orchestration of a central server. In this setup, each client's data is private to itself and is not transferable to other clients or the server. Though FL paradigm has received significant interest recently from the research community, the problem of selecting the relevant clients w.r.t. the central server's learning objective is under-explored. We refer to these problems as Federated Relevant Client Selection (FRCS). Because the server doesn't have…
History-Aware and Dynamic Client Contribution in Federated Learning
Max Planck Institute for Software Systems, Germany（马克斯·普朗克软件系统研究所，德国）；Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 - CRIStAL, France（里尔大学，法国）；Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), Singapore（高性能计算研究所（IHPC），新加坡）；EVYD Technology, Singapore（EVYD技术公司，新加坡）
Addressing Data Quality Decompensation in Federated Learning via Dynamic Client Selection
Department of Computer Science and Artificial Intelligence, Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI), University of Granada, Spain（计算机科学与人工智能系，数据科学与计算智能安达卢西亚研究机构（DaSCI），格拉纳达大学，西班牙）；Department of Software Engineering, Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI), University of Granada, Spain（软件工程系，数据科学与计算智能安达卢西亚研究机构（DaSCI），格拉纳达大学，西班牙）；School of Management, Hangzhou Dianzi University, China（管理学院，杭州电子科技大学，中国）；Shaanxi Key Laboratory of Information Communication Network and Security, Xi’an University of Posts & Telecommunications, China（陕西省信息通信网络与安全重点实验室，西安邮电大学，中国）
Owen Sampling Accelerates Contribution Estimation in Federated Learning
Department of Electrical and Software Engineering, University of Calgary, Calgary, Alberta, Canada（电气与软件工程系，卡里尔大学）；Department of Electrical Engineering and Computer Science, York University, Toronto, Ontario, Canada（电气工程与计算机科学系，约克大学）；School of Information, University of Michigan, Ann Arbor, Michigan, USA（信息学院，密歇根大学）
Federated learning (FL) is a popular technique to train machine learning (ML) models on decentralized data sources. In order to sustain long-term participation of data owners, it is important to fairly appraise each data source and compensate data owners for their contribution to the training process. The Shapley value (SV) defines a unique payoff scheme that satisfies many desiderata for a data value notion. It has been increasingly used for valuing training data in centralized learning. However, computing the SV requires exhaustively evaluating the model performance on every subset of data s…
Federated learning (FL) is an emerging collaborative machine learning method to train models on distributed datasets with privacy concerns. To properly incentivize data owners to contribute their efforts, Shapley Value (SV) is often adopted to fairly assess their contribution. However, the calculation of SV is time-consuming and computationally costly. In this paper, we propose FedCoin, a blockchain-based peer-to-peer payment system for FL to enable a feasible SV based profit distribution. In FedCoin, blockchain consensus entities calculate SVs and a new block is created based on the proof of …
Swiss Re（瑞士再保险）；Ping An P&C Insurance Co., Ltd.（平安财产保险股份有限公司）
Federated Machine Learning (FML) creates an ecosystem for multiple parties to collaborate on building models while protecting data privacy for the participants. A measure of the contribution for each party in FML enables fair credits allocation. In this paper we develop simple but powerful techniques to fairly calculate the contributions of multiple parties in FML, in the context of both horizontal FML and vertical FML. For Horizontal FML we use deletion method to calculate the grouped instance influence. For Vertical FML we use Shapley Values to calculate the grouped feature importance. Our m…
Private and Robust Contribution Evaluation in Federated Learning
University of La Laguna（拉拉加大学）；HUN-REN Hungarian Research Network（HUN-REN匈牙利研究网络）；Chalmers University（查尔姆斯大学）；Budapest University of Technology and Economics（布达佩斯技术经济大学）
Northeastern University（东北大学）；HKUST（香港科技大学）；Hong Kong Baptist University（香港浸会大学）
Federated learning utilizes various resources provided by participants to collaboratively train a global model, which potentially address the data privacy issue of machine learning. In such promising paradigm, the performance will be deteriorated without sufficient training data and other resources in the learning process. Thus, it is quite crucial to inspire more participants to contribute their valuable resources with some payments for federated learning. In this paper, we present a comprehensive survey of incentive schemes for federate learning. Specifically, we identify the incentive probl…
Federated Learning is a promising machine learning paradigm when multiple parties collaborate to build a high-quality machine learning model. Nonetheless, these parties are only willing to participate when given enough incentives, such as a fair reward based on their contributions. Many studies explored Shapley value based methods to evaluate each party's contribution to the learned model. However, they commonly assume a semi-trusted server to train the model and evaluate the data owners' model contributions, which lacks transparency and may hinder the success of federated learning in practice…
The Hong Kong Polytechnic University（香港理工大学）；Beijing Institute of Technology（北京理工大学）
Facing the challenge of statistical diversity in client local data distribution, personalized federated learning (PFL) has become a growing research hotspot. Although the state-of-the-art methods with model similarity-based pairwise collaboration have achieved promising performance, they neglect the fact that model aggregation is essentially a collaboration process within the coalition, where the complex multiwise influences take place among clients. In this paper, we first apply Shapley value (SV) from coalition game theory into the PFL scenario. To measure the multiwise collaboration among a…
Federated Learning for Global Carbon Emission Forecasting: A Hybrid Time-Series Approach with Statistical and Neural Models
School of Computing and Artificial Intelligence, Southwest Jiaotong University（西南交通大学计算机与人工智能学院）；Department of Computer Science, Bahria University（巴里亚大学计算机科学系）；School of Mechanical Engineering, Southwest Jiaotong University（西南交通大学机械工程学院）
Cost-TrustFL: Cost-Aware Hierarchical Federated Learning with Lightweight Reputation Evaluation across Multi-Cloud
Westcliff University（韦斯特克莱夫大学）；University of Virginia（弗吉尼亚大学）；University of Washington（华盛顿大学）；Northeastern University（东北大学）；University of Michigan（密歇根大学）
Towards Explainable Federated Learning: Understanding the Impact of Differential Privacy
1 Institute of Exact；Natural Sciences, Federal University of Par\'a, Brazil；Resource Optimization in Networks -- PROTON Lab School of Electrical, Computer；Energy Engineering, Arizona State University, Tempe, AZ, USA；3 Faculty of Mathematics \& Science, Brock University, Canada
Incentives that compensate for the involved costs in the decentralized training of a Federated Learning (FL) model act as a key stimulus for clients' long-term participation. However, it is challenging to convince clients for quality participation in FL due to the absence of: (i) full information on the client's data quality and properties; (ii) the value of client's data contributions; and (iii) the trusted mechanism for monetary incentive offers. This often leads to poor efficiency in training and communication. While several works focus on strategic incentive designs and client selection to…
On-device Content-based Recommendation with Single-shot Embedding Pruning: A Cooperative Game Perspective
The University of Queensland（昆士兰大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Griffith University（格里菲斯大学）；University of Electronic Science and Technology of China（电子科技大学）
Efficient Data Valuation Approximation in Federated Learning: A Sampling-based Approach
State Key Laboratory of Complex & Critical Software Environment Lab, School of Computer Science and Engineering, Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University, China（复杂与关键软件环境实验室，计算机科学与工程学院，未来区块链与隐私计算先进创新中心，北京航空航天大学）
Communication-Efficient Multimodal Federated Learning: Joint Modality and Client Selection
School of Electrical and Computer Engineering, Purdue University（普渡大学电气与计算机工程学院）；Department of Computer Science and Engineering, Yonsei University（延世大学计算机科学与工程系）；School of Electrical and Computer Engineering, Princeton University（普林斯顿大学电气与计算机工程学院）；Saab Inc.（Saab公司）
Northeast Electric Power University（东北电力大学）；State Grid Jilin Electric Power Co., Ltd. Jilin Power Supply Company（国网吉林省电力有限公司吉林供电公司）；Huazhong University of Science and Technology（华中科技大学）；Naval University of Engineering（海军工程大学）；Cranfield University（克兰菲尔德大学）
In a modern power system with an increasing proportion of renewable energy, wind power prediction is crucial to the arrangement of power grid dispatching plans due to the volatility of wind power. However, traditional centralized forecasting methods raise concerns regarding data privacy-preserving and data islands problem. To handle the data privacy and openness, we propose a forecasting scheme that combines federated learning and deep reinforcement learning (DRL) for ultra-short-term wind power forecasting, called federated deep reinforcement learning (FedDRL). Firstly, this paper uses the de…
As Machine Learning (ML) models are becoming increasingly complex, one of the central challenges is their deployment at scale, such that companies and organizations can create value through Artificial Intelligence (AI). An emerging paradigm in ML is a federated approach where the learning model is delivered to a group of heterogeneous agents partially, allowing agents to train the model locally with their own data. However, the problem of valuation of models, as well the questions of incentives for collaborative training and trading of data/models, have received limited treatment in the litera…
