In many real life situations, including job and loan applications, gatekeepers must make justified and fair real-time decisions about a person's fitness for a particular opportunity. In this paper, we aim to accomplish approximate group fairness in an online stochastic decision-making process, where the fairness metric we consider is equalized odds. Our work follows from the classical learning-from-experts scheme, assuming a finite set of classifiers (human experts, rules, options, etc) that cannot be modified. We run separate instances of the algorithm for each label class as well as sensitive groups, where the probability of choosing each instance is optimized for both fairness and regret. Our theoretical results show that approximately equalized odds can be achieved without sacrificing much regret. We also demonstrate the performance of the algorithm on real data sets commonly used by the fairness community.
Yale University（耶鲁大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Amazon AWS AI（亚马逊云科技人工智能）
Machine learning models are often implemented in cohort with humans in the pipeline, with the model having an option to defer to a domain expert in cases where it has low confidence in its inference. Our goal is to design mechanisms for ensuring accuracy and fairness in such prediction systems that combine machine learning model inferences and domain expert predictions. Prior work on "deferral systems" in classification settings has focused on the setting of a pipeline with a single expert and aimed to accommodate the inaccuracies and biases of this expert to simultaneously learn an inference …
In-Context Impersonation Reveals Large Language Models' Strengths and Biases
University of Tübingen（图宾根大学）；Tübingen AI Center（图宾根人工智能中心）；University of Porto（波尔图大学）；INESC TEC（葡萄牙系统与计算机工程研究所（技术与科学））；Max Planck Institute for Biological Cybernetics（马克斯·普朗克生物控制论研究所）
Sorbonne Université（索邦大学）；CNRS（法国国家科学研究中心）；LIP6（巴黎第六大学计算机科学实验室）；Facebook AI Research（脸书人工智能研究院）；Georgia Institute of Technology（佐治亚理工学院）
Visual Question Answering (VQA) is the task of answering questions about an image. Some VQA models often exploit unimodal biases to provide the correct answer without using the image information. As a result, they suffer from a huge drop in performance when evaluated on data outside their training set distribution. This critical issue makes them unsuitable for real-world settings. We propose RUBi, a new learning strategy to reduce biases in any VQA model. It reduces the importance of the most biased examples, i.e. examples that can be correctly classified without looking at the image. It impli…
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Max Planck Institute for Software Systems（马克斯·普朗克软件系统研究所）
Societies often rely on human experts to take a wide variety of decisions affecting their members, from jail-or-release decisions taken by judges and stop-and-frisk decisions taken by police officers to accept-or-reject decisions taken by academics. In this context, each decision is taken by an expert who is typically chosen uniformly at random from a pool of experts. However, these decisions may be imperfect due to limited experience, implicit biases, or faulty probabilistic reasoning. Can we improve the accuracy and fairness of the overall decision making process by optimizing the assignment…
Université Libre de Bruxelles（布鲁塞尔自由大学）；Vrije Universiteit Brussel（布鲁塞尔自由大学）；UC Berkeley（加州大学伯克利分校）；National Research Council（国家研究委员会）
Quite some real-world problems can be formulated as decision-making problems wherein one must repeatedly make an appropriate choice from a set of alternatives. Multiple expert judgements, whether human or artificial, can help in taking correct decisions, especially when exploration of alternative solutions is costly. As expert opinions might deviate, the problem of finding the right alternative can be approached as a collective decision making problem (CDM) via aggregation of independent judgements. Current state-of-the-art approaches focus on efficiently finding the optimal expert, and thus p…
Scuola Normale Superiore（比萨高等师范学院）；ISI Foundation（ISI基金会）；University of Pisa（比萨大学）
The pervasive application of algorithmic decision-making is raising concerns on the risk of unintended bias in AI systems deployed in critical settings such as healthcare. The detection and mitigation of biased models is a very delicate task which should be tackled with care and involving domain experts in the loop. In this paper we introduce FairLens, a methodology for discovering and explaining biases. We show how our tool can be used to audit a fictional commercial black-box model acting as a clinical decision support system. In this scenario, the healthcare facility experts can use FairLen…
Demographic Bias of Expert-Level Vision-Language Foundation Models in Medical Imaging
Massachusetts Institute of Technology（麻省理工学院）；University of California, San Diego（加州大学圣地亚哥分校）；University of Washington（华盛顿大学）；University of Washington School of Medicine（华盛顿大学医学院）
Boston University（波士顿大学）；Microsoft Research New England（微软新英格兰研究院）
Machine learning algorithms are optimized to model statistical properties of the training data. If the input data reflects stereotypes and biases of the broader society, then the output of the learning algorithm also captures these stereotypes. In this paper, we initiate the study of gender stereotypes in {\em word embedding}, a popular framework to represent text data. As their use becomes increasingly common, applications can inadvertently amplify unwanted stereotypes. We show across multiple datasets that the embeddings contain significant gender stereotypes, especially with regard to profe…
Machine learning models learn what we teach them to learn. Machine learning is at the heart of recommender systems. If a machine learning model is trained on biased data, the resulting recommender system may reflect the biases in its recommendations. Biases arise at different stages in a recommender system, from existing societal biases in the data such as the professional gender gap, to biases introduced by the data collection or modeling processes. These biases impact the performance of various components of recommender systems, from offline training, to evaluation and online serving of reco…
Yale University（耶鲁大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Amazon AWS AI（亚马逊云服务人工智能）；Fiddler AI
In hybrid human-machine deferral frameworks, a classifier can defer uncertain cases to human decision-makers (who are often themselves fallible). Prior work on simultaneous training of such classifier and deferral models has typically assumed access to an oracle during training to obtain true class labels for training samples, but in practice there often is no such oracle. In contrast, we consider a "closed" decision-making pipeline in which the same fallible human decision-makers used in deferral also provide training labels. How can imperfect and biased human expert labels be used to train a…
Sapienza University of Rome（罗马大学）；Stevens Institute of Technology（史蒂文斯理工学院）；Institute of Applied Computing CNR（意大利国家研究委员会应用计算研究所）
Password security hinges on an in-depth understanding of the techniques adopted by attackers. Unfortunately, real-world adversaries resort to pragmatic guessing strategies such as dictionary attacks that are inherently difficult to model in password security studies. In order to be representative of the actual threat, dictionary attacks must be thoughtfully configured and tuned. However, this process requires a domain-knowledge and expertise that cannot be easily replicated. The consequence of inaccurately calibrating dictionary attacks is the unreliability of password security analyses, impai…
