We consider the dataset valuation problem, that is, the problem of quantifying the incremental gain, to some relevant pre-defined utility of a machine learning task, of aggregating an individual dataset to others. The Shapley value is a natural tool to perform dataset valuation due to its formal axiomatic justification, which can be combined with Monte Carlo integration to overcome the computational tractability challenges. Such generic approximation methods, however, remain expensive in some cases. In this paper, we exploit the knowledge about the structure of the dataset valuation problem to devise more efficient Shapley value estimators. We propose a novel approximation, referred to as discrete uniform Shapley, which is expressed as an expectation under a discrete uniform distribution with support of reasonable size. We justify the relevancy of the proposed framework via asymptotic and non-asymptotic theoretical guarantees and illustrate its benefits via an extensive set of numerical experiments.
As data becomes the fuel driving technological and economic growth, a fundamental challenge is how to quantify the value of data in algorithmic predictions and decisions. For example, in healthcare and consumer markets, it has been suggested that individuals should be compensated for the data that they generate, but it is not clear what is an equitable valuation for individual data. In this work, we develop a principled framework to address data valuation in the context of supervised machine learning. Given a learning algorithm trained on $n$ data points to produce a predictor, we propose data…
Data Valuation by Fusing Global and Local Statistical Information
Peking University（北京大学）；University of Chinese Academy of Sciences（中国科学院大学）；Hong Kong Baptist University（香港 Baptist大学）；Renmin University of China（中国人民大学）
Proxy-Based Approximation of Shapley and Banzhaf Interactions
LMU Munich（慕尼黑大学）；MCML；DFKI（德意志联邦防务研究院）；Centre for Credible AI, Warsaw University of Technology（华沙技术大学可信AI中心）；University of Warsaw（华沙大学）；Claremont McKenna College（克莱尔蒙特麦肯纳学院）；Bielefeld University（比勒菲尔德大学）
AstraZeneca（阿斯利康）；The University of Edinburgh（爱丁堡大学）；Toulouse School of Economics（图卢兹经济学院）；Institute for Advanced Study in Toulouse（图卢兹高等研究院）；Central European University（中欧大学）
Over the last few years, the Shapley value, a solution concept from cooperative game theory, has found numerous applications in machine learning. In this paper, we first discuss fundamental concepts of cooperative game theory and axiomatic properties of the Shapley value. Then we give an overview of the most important applications of the Shapley value in machine learning: feature selection, explainability, multi-agent reinforcement learning, ensemble pruning, and data valuation. We examine the most crucial limitations of the Shapley value and point out directions for future research.
Massachusetts General Hospital（麻省总医院）；Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）
Federated Learning (FL) wherein multiple institutions collaboratively train a machine learning model without sharing data is becoming popular. Participating institutions might not contribute equally, some contribute more data, some better quality data or some more diverse data. To fairly rank the contribution of different institutions, Shapley value (SV) has emerged as the method of choice. Exact SV computation is impossibly expensive, especially when there are hundreds of contributors. Existing SV computation techniques use approximations. However, in healthcare where the number of contributi…
What is the value of an individual model in an ensemble of binary classifiers? We answer this question by introducing a class of transferable utility cooperative games called \textit{ensemble games}. In machine learning ensembles, pre-trained models cooperate to make classification decisions. To quantify the importance of models in these ensemble games, we define \textit{Troupe} -- an efficient algorithm which allocates payoffs based on approximate Shapley values of the classifiers. We argue that the Shapley value of models in these games is an effective decision metric for choosing a high per…
Department of Statistics, LMU Munich（LMU慕尼黑统计系）；Department of Electrical Engineering and Computer Science, UC Berkeley（伯克利电子工程与计算机科学系）；Mathematical Sciences Department, Claremont McKenna College（克莱门茨麦肯纳学院数学科学系）
ETH Zurich（苏黎世联邦理工学院）；Microsoft（微软）；UIUC（伊利诺伊大学厄巴纳-香槟分校）；University of Amsterdam（阿姆斯特丹大学）
Developing modern machine learning (ML) applications is data-centric, of which one fundamental challenge is to understand the influence of data quality to ML training -- "Which training examples are 'guilty' in making the trained ML model predictions inaccurate or unfair?" Modeling data influence for ML training has attracted intensive interest over the last decade, and one popular framework is to compute the Shapley value of each training example with respect to utilities such as validation accuracy and fairness of the trained ML model. Unfortunately, despite recent intensive interest and res…
LMU Munich（慕尼黑大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；University of Warsaw（华沙大学）；Warsaw University of Technology（华沙理工大学）；Bielefeld University（比勒费尔德大学）；CITEC（认知交互技术中心）；Paderborn University（帕德博恩大学）
Guixu: Valuation-Driven Data Discovery for Autonomous AI Agents with On-Chain Attestation
Zhejiang University（浙江大学）；The State Key Laboratory of Blockchain and Data Security（区块链与数据安全国家重点实验室）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新区（滨江）区块链研究院）
Incentivizing Truthfulness and Collaborative Fairness in Bayesian Learning
Department of Computer Science, National University of Singapore, Singapore（新加坡国立大学计算机科学系）；Research (A STAR), Singapore（新加坡A*STAR研究）；Department of Electrical Engineering（电气工程系）；Computer Science, Massachusetts Institute of Technology, USA（美国麻省理工学院计算机科学系）
