Bolukbasi et al. (2016) presents one of the first gender bias mitigation techniques for word representations. Their method takes pre-trained word representations as input and attempts to isolate a linear subspace that captures most of the gender bias in the representations. As judged by an analogical evaluation task, their method virtually eliminates gender bias in the representations. However, an implicit and untested assumption of their method is that the bias subspace is actually linear. In this work, we generalize their method to a kernelized, nonlinear version. We take inspiration from kernel principal component analysis and derive a nonlinear bias isolation technique. We discuss and overcome some of the practical drawbacks of our method for non-linear gender bias mitigation in word representations and analyze empirically whether the bias subspace is actually linear. Our analysis shows that gender bias is in fact well captured by a linear subspace, justifying the assumption of Bolukbasi et al. (2016).
Pre-trained language models encode undesirable social biases, which are further exacerbated in downstream use. To this end, we propose MABEL (a Method for Attenuating Gender Bias using Entailment Labels), an intermediate pre-training approach for mitigating gender bias in contextualized representations. Key to our approach is the use of a contrastive learning objective on counterfactually augmented, gender-balanced entailment pairs from natural language inference (NLI) datasets. We also introduce an alignment regularizer that pulls identical entailment pairs along opposite gender directions cl…
Beijing Academy of Artificial Intelligence（北京人工智能研究院）；The University of Sheffield（谢菲尔德大学）；University of Michigan（密歇根大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）；Sichuan University（四川大学）；McGill University（麦吉尔大学）
As natural language processing (NLP) for gender bias becomes a significant interdisciplinary topic, the prevalent data-driven techniques such as large-scale language models suffer from data inadequacy and biased corpus, especially for languages with insufficient resources such as Chinese. To this end, we propose a Chinese cOrpus foR Gender bIas Probing and Mitigation CORGI-PM, which contains 32.9k sentences with high-quality labels derived by following an annotation scheme specifically developed for gender bias in the Chinese context. Moreover, we address three challenges for automatic textual…
University of Virginia（弗吉尼亚大学）；Salesforce Research（赛富时研究院）
Word embeddings derived from human-generated corpora inherit strong gender bias which can be further amplified by downstream models. Some commonly adopted debiasing approaches, including the seminal Hard Debias algorithm, apply post-processing procedures that project pre-trained word embeddings into a subspace orthogonal to an inferred gender subspace. We discover that semantic-agnostic corpus regularities such as word frequency captured by the word embeddings negatively impact the performance of these algorithms. We propose a simple but effective technique, Double Hard Debias, which purifies …
Machine Learning models have been deployed across many different aspects of society, often in situations that affect social welfare. Although these models offer streamlined solutions to large problems, they may contain biases and treat groups or individuals unfairly based on protected attributes such as gender. In this paper, we introduce several examples of machine learning gender bias in practice followed by formalizations of fairness. We provide a survey of fairness research by detailing influential pre-processing, in-processing, and post-processing bias mitigation algorithms. We then propo…
In this study, we proposed a convolutional neural network model for gender prediction using English Twitter text as input. Ensemble of proposed model achieved an accuracy at 0.8237 on gender prediction and compared favorably with the state-of-the-art performance in a recent author profiling task. We further leveraged the trained models to predict the gender labels from an HPV vaccine related corpus and identified gender difference in public perceptions regarding HPV vaccine. The findings are largely consistent with previous survey-based studies.
Visa Inc.（Visa公司）；Harvard John A. Paulson School Of Engineering And Applied Sciences（哈佛大学约翰·A·保尔森工程与应用科学学院）；FM: Systems（FM: Systems公司）；Autodesk（Autodesk公司）
Gender classification algorithms have important applications in many domains today such as demographic research, law enforcement, as well as human-computer interaction. Recent research showed that algorithms trained on biased benchmark databases could result in algorithmic bias. However, to date, little research has been carried out on gender classification algorithms' bias towards gender minorities subgroups, such as the LGBTQ and the non-binary population, who have distinct characteristics in gender expression. In this paper, we began by conducting surveys on existing benchmark databases for…
Published studies have suggested the bias of automated face-based gender classification algorithms across gender-race groups. Specifically, unequal accuracy rates were obtained for women and dark-skinned people. To mitigate the bias of gender classifiers, the vision community has developed several strategies. However, the efficacy of these mitigation strategies is demonstrated for a limited number of races mostly, Caucasian and African-American. Further, these strategies often offer a trade-off between bias and classification accuracy. To further advance the state-of-the-art, we leverage the p…
Sex and gender-based healthcare disparities contribute to differences in health outcomes. We focus on time to diagnosis (TTD) by conducting two large-scale, complementary analyses among men and women across 29 phenotypes and 195K patients. We first find that women are consistently more likely to experience a longer TTD than men, even when presenting with the same conditions. We further explore how TTD disparities affect diagnostic performance between genders, both across and persistent to time, by evaluating gender-agnostic disease classifiers across increasing diagnostic information. In both …
Exploring Gender-Specific Speech Patterns in Automatic Suicide Risk Assessment
University of Augsburg（奥格斯堡大学）；TU Munich（慕尼黑工业大学）；District Hospital Augsburg（奥格斯堡地区医院）；German Center for Mental Health（德国心理健康中心）；Imperial College（帝国理工学院）
Manipal Institute of Technology（马尼帕尔理工学院）；MAHE（马尼帕尔高等教育学院）
With language models being deployed increasingly in the real world, it is essential to address the issue of the fairness of their outputs. The word embedding representations of these language models often implicitly draw unwanted associations that form a social bias within the model. The nature of gendered languages like Hindi, poses an additional problem to the quantification and mitigation of bias, owing to the change in the form of the words in the sentence, based on the gender of the subject. Additionally, there is sparse work done in the realm of measuring and debiasing systems for Indic …
Explanatory Interactive Machine Learning for Bias Mitigation in Visual Gender Classification
Johannes Kepler University Linz（约翰内斯·开普勒大学林茨）；Institute of Creative Media Technologies（创意媒体技术研究所）；St. Pölten University of Applied Sciences（施蒂利克大学应用科学学院）；Institute of Computational Perception（计算感知研究所）
A major problem with using automated classification systems is that if they are not engineered correctly and with fairness considerations, they could be detrimental to certain populations. Furthermore, while engineers have developed cutting-edge technologies for image classification, there is still a gap in the application of these models in human heritage collections, where data sets usually consist of low-quality pictures of people with diverse ethnicity, gender, and age. In this work, we evaluate three bias mitigation techniques using two state-of-the-art neural networks, Xception and Effic…
Towards Trustworthy Audio Deepfake Detection: A Systematic Framework for Diagnosing and Mitigating Gender Bias
School of Computing, Wichita State University（维克托利亚州立大学计算机学院）；Institut national de la recherche scientifique (INRS-EMT)（国家科学研究机构（INRS-EMT））；INRS-UQO Mixed Research Unit on Cybersecurity（INRS-UQO网络安全混合研究单位）
GABInsight: Exploring Gender-Activity Binding Bias in Vision-Language Models
Sharif University of Technology（谢里夫理工大学）；Shamsipour Technical and Vocational College（沙姆西普尔技术与职业学院）；ecole polytechnique fédérale de lausanne (EPFL)（洛桑联邦理工学院）；IDIAP research institute（IDIAP研究所）
Comparative assessment of fairness definitions and bias mitigation strategies in machine learning-based diagnosis of Alzheimer's disease from MR images
National Technical University of Athens（雅典国家技术大学）；University of Pennsylvania（宾夕法尼亚大学）
Tsinghua University（清华大学）；University of California, Los Angeles（加州大学洛杉矶分校）
Advanced machine learning techniques have boosted the performance of natural language processing. Nevertheless, recent studies, e.g., Zhao et al. (2017) show that these techniques inadvertently capture the societal bias hidden in the corpus and further amplify it. However, their analysis is conducted only on models' top predictions. In this paper, we investigate the gender bias amplification issue from the distribution perspective and demonstrate that the bias is amplified in the view of predicted probability distribution over labels. We further propose a bias mitigation approach based on post…
Causally Testing Gender Bias in LLMs: A Case Study on Occupational Bias
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Columbia University（哥伦比亚大学）；RWTH Aachen（亚琛工业大学）；University of Michigan（密歇根大学）；ETH Zürich（苏黎世联邦理工学院）；Max Planck Institute（马克斯·普朗克研究所）
Exploring Potential Prompt Injection Attacks in Federated Military LLMs and Their Mitigation
Institute of Information & Communications Technology Planning & Evaluation (IITP)-ITRC (Information Technology Research Center)（信息与通信技术规划与评估机构（IITP）-ITRC（信息技术研究中心））
Vrije Universiteit Brussel（布鲁塞尔自由大学）；National University of Ireland Galway（爱尔兰国立大学戈尔韦分校）；University of Antwerp（安特卫普大学）；Hasselt University（哈瑟尔特大学）；HU University of Applied Sciences Utrecht（乌得勒支应用科学大学）
Infectious disease outbreaks can have a disruptive impact on public health and societal processes. As decision making in the context of epidemic mitigation is hard, reinforcement learning provides a methodology to automatically learn prevention strategies in combination with complex epidemic models. Current research focuses on optimizing policies w.r.t. a single objective, such as the pathogen's attack rate. However, as the mitigation of epidemics involves distinct, and possibly conflicting criteria (i.a., prevalence, mortality, morbidity, cost), a multi-objective approach is warranted to lear…
A Typology for Exploring the Mitigation of Shortcut Behavior
Technical University of Darmstadt（达姆施塔特工业大学）；Hessian Center for Artificial Intelligence (hessian.AI)（黑森人工智能中心）；LAION；German Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Centre for Cognitive Science（认知科学中心）
