With an increased focus on incorporating fairness in machine learning models, it becomes imperative not only to assess and mitigate bias at each stage of the machine learning pipeline but also to understand the downstream impacts of bias across stages. Here we consider a general, but realistic, scenario in which a predictive model is learned from (potentially biased) training data, and model predictions are assessed post-hoc for fairness by some auditing method. We provide a theoretical analysis of how a specific form of data bias, differential sampling bias, propagates from the data stage to the prediction stage. Unlike prior work, we evaluate the downstream impacts of data biases quantitatively rather than qualitatively and prove theoretical guarantees for detection. Under reasonable assumptions, we quantify how the amount of bias in the model predictions varies as a function of the amount of differential sampling bias in the data, and at what point this bias becomes provably detectable by the auditor. Through experiments on two criminal justice datasets -- the well-known COMPAS dataset and historical data from NYPD's stop and frisk policy -- we demonstrate that the theoretical results hold in practice even when our assumptions are relaxed.
Understanding the implicit bias of training algorithms is of crucial importance in order to explain the success of overparametrised neural networks. In this paper, we study the dynamics of stochastic gradient descent over diagonal linear networks through its continuous time version, namely stochastic gradient flow. We explicitly characterise the solution chosen by the stochastic flow and prove that it always enjoys better generalisation properties than that of gradient flow. Quite surprisingly, we show that the convergence speed of the training loss controls the magnitude of the biasing effect…
Provable Low-Frequency Bias of In-Context Learning of Representations
Computer Science and Engineering, University of Michigan（密歇根大学计算机科学与工程系）；CBS-NTT Physics of Intelligence Program, Harvard University（哈佛大学CBS-NTT智能物理计划）；Physics of Artificial Intelligence Group, NTT Research, Inc.（NTT研究公司人工智能物理小组）
Machine learning models are increasingly deployed for critical decision-making tasks, making it important to verify that they do not contain gender or racial biases picked up from training data. Typical approaches to achieve fairness revolve around efforts to clean or curate training data, with post-hoc statistical evaluation of the fairness of the model on evaluation data. In contrast, we propose techniques to \emph{prove} fairness using recently developed formal methods that verify properties of neural network models.Beyond the strength of guarantee implied by a formal proof, our methods hav…
Mathematical Institute for Data Science（数据科学数学研究所）；Johns Hopkins University（约翰斯·霍普金斯大学）
Robust subspace recovery (RSR) is a fundamental problem in robust representation learning. Here we focus on a recently proposed RSR method termed Dual Principal Component Pursuit (DPCP) approach, which aims to recover a basis of the orthogonal complement of the subspace and is amenable to handling subspaces of high relative dimension. Prior work has shown that DPCP can provably recover the correct subspace in the presence of outliers, as long as the true dimension of the subspace is known. We show that DPCP can provably solve RSR problems in the {\it unknown} subspace dimension regime, as long…
The use of machine learning models in consequential decision making often exacerbates societal inequity, in particular yielding disparate impact on members of marginalized groups defined by race and gender. The area under the ROC curve (AUC) is widely used to evaluate the performance of a scoring function in machine learning, but is studied in algorithmic fairness less than other performance metrics. Due to the pairwise nature of the AUC, defining an AUC-based group fairness metric is pairwise-dependent and may involve both \emph{intra-group} and \emph{inter-group} AUCs. Importantly, consideri…
Provable and scalable quantum Gaussian processes for quantum learning
Theoretical Division, Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室理论部）；Department of Computer Science（计算机科学系）；Institute of Applied Mathematics, University of British Columbia（不列颠哥伦比亚大学应用数学学院）；Stewart Blusson Quantum Matter Institute（斯图尔特·布卢森量子物质研究所）；Information Sciences, Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室信息科学部）；Donostia International Physics Center（多诺西亚国际物理中心）；Department of Applied Physics, Gipuzkoa School of Engineering, University of the Basque Country (UPV/EHU)（巴斯克国家大学（UPV/EHU）吉普兹科亚工程学院应用物理系）；Biophysics, Doctoral School, Universidad Autónoma de Madrid（马德里自治大学生物物理系博士学院）；Department for Quantum Information（量子信息部）；Computation at Kepler (QUICK), Johannes Kepler University（开普勒大学量子信息与计算部（QUICK））；Quantum Science Center, Oak Ridge, TN 37931, USA（橡树岭量子科学中心）
Fairis: Fairness-Aware Aggregation with Provable Influence Containment against Fairness Poisoning Attacks in Collaborative Machine Learning
Bowie State University（鲍伊州立大学）；Télécom SudParis（南巴黎电信学院）；Institut Polytechnique de Paris（巴黎综合理工学院）；Brown University（布朗大学）；Northern Virginia Community College（北弗吉尼亚社区学院）
Department of Computer Science IIIT Delhi, India（印度德里IIIT计算机科学系）；Microsoft Research India（微软印度研究院）；Department of Computer Science and Automation Indian Institute of Science, Bengaluru, India（印度班加罗尔印度科学学院计算机科学与自动化系）
