As Machine Learning models continue to be relied upon for making automated decisions, the issue of model bias becomes more and more prevalent. In this paper, we approach training a text classifica-tion model and optimize on bias minimization by measuring not only the models performance on our dataset as a whole, but also how it performs across different subgroups. This requires measuring per-formance independently for different demographic subgroups and measuring bias by comparing them to results from the rest of our data. We show how unintended bias can be detected using these metrics and how removing bias from a dataset completely can result in worse results.
University of Toronto（多伦多大学）；Vector Institute of Artificial Intelligence（向量人工智能研究院）；Unity Health Toronto（多伦多统一健康中心）
Existing studies have investigated the tendency of autoregressive language models to generate contexts that exhibit undesired biases and toxicity. Various debiasing approaches have been proposed, which are primarily categorized into data-based and decoding-based. In our study, we investigate the ensemble of the two debiasing paradigms, proposing to use toxic corpus as an additional resource to reduce the toxicity. Our result shows that toxic corpus can indeed help to reduce the toxicity of the language generation process substantially, complementing the existing debiasing methods.
Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation
Institute for Logic, Language and Computation, University of Amsterdam（逻辑、语言与计算研究所，阿姆斯特丹大学）；Center for Language and Cognition, University of Groningen（语言与认知中心，格罗宁根大学）
National Taiwan University（台湾大学）；MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；Amazon AI（亚马逊人工智能）
Automatic detection of toxic language plays an essential role in protecting social media users, especially minority groups, from verbal abuse. However, biases toward some attributes, including gender, race, and dialect, exist in most training datasets for toxicity detection. The biases make the learned models unfair and can even exacerbate the marginalization of people. Considering that current debiasing methods for general natural language understanding tasks cannot effectively mitigate the biases in the toxicity detectors, we propose to use invariant rationalization (InvRat), a game-theoreti…
University of Illinois-Urbana Champaign（伊利诺伊大学厄巴纳-香槟分校）
Previous work has examined how debiasing language models affect downstream tasks, specifically, how debiasing techniques influence task performance and whether debiased models also make impartial predictions in downstream tasks or not. However, what we don't understand well yet is why debiasing methods have varying impacts on downstream tasks and how debiasing techniques affect internal components of language models, i.e., neurons, layers, and attentions. In this paper, we decompose the internal mechanisms of debiasing language models with respect to gender by applying causal mediation analysi…
Center for Information and Language Processing (CIS), LMU Munich（慕尼黑大学信息与语言处理中心）；Institute of Social and Cultural Anthropology, LMU Munich（慕尼黑大学社会与文化人类学研究所）
When trained on large, unfiltered crawls from the internet, language models pick up and reproduce all kinds of undesirable biases that can be found in the data: they often generate racist, sexist, violent or otherwise toxic language. As large models require millions of training examples to achieve good performance, it is difficult to completely prevent them from being exposed to such content. In this paper, we first demonstrate a surprising finding: pretrained language models recognize, to a considerable degree, their undesirable biases and the toxicity of the content they produce. We refer to…
Take its Essence, Discard its Dross! Debiasing for Toxic Language Detection via Counterfactual Causal Effect
School of Computer Science and Technology, Dalian University of Technology（大连理工大学计算机科学与技术学院）；School of Foreign Chinese, Dalian University of Technology（大连理工大学对外汉语学院）
Preconditioned Test-Time Adaptation for Out-of-Distribution Debiasing in Narrative Generation
Laboratory for Artificial Intelligence in Mathematics Education, Stevens Institute of Technology（数学教育中的人工智能实验室，史蒂文斯理工学院）；Independent Researcher（独立研究者）；NLP2CT Lab, Department of Computer and Information Science, University of Macau（NLP2CT实验室，澳门大学计算机与信息科学系）
Harbin Institute of Technology（哈尔滨工业大学）；Tencent CSIG（腾讯云与智慧产业事业群）
With the recent proliferation of the use of text classifications, researchers have found that there are certain unintended biases in text classification datasets. For example, texts containing some demographic identity-terms (e.g., "gay", "black") are more likely to be abusive in existing abusive language detection datasets. As a result, models trained with these datasets may consider sentences like "She makes me happy to be gay" as abusive simply because of the word "gay." In this paper, we formalize the unintended biases in text classification datasets as a kind of selection bias from the no…
University of Illinois Chicago（伊利诺伊大学芝加哥分校）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Arizona State University（亚利桑那州立大学）
Natural Language Processing (NLP) models have been found discriminative against groups of different social identities such as gender and race. With the negative consequences of these undesired biases, researchers have responded with unprecedented effort and proposed promising approaches for bias mitigation. In spite of considerable practical importance, current algorithmic fairness literature lacks an in-depth understanding of the relations between different forms of biases. Social bias is complex by nature. Numerous studies in social psychology identify the "generalized prejudice", i.e., gene…
College of Information Science and Technology（信息科学与技术学院）；Pennsylvania State University（宾夕法尼亚州立大学）
Sociodemographic biases are a common problem for natural language processing, affecting the fairness and integrity of its applications. Within sentiment analysis, these biases may undermine sentiment predictions for texts that mention personal attributes that unbiased human readers would consider neutral. Such discrimination can have great consequences in the applications of sentiment analysis both in the public and private sectors. For example, incorrect inferences in applications like online abuse and opinion analysis in social media platforms can lead to unwanted ramifications, such as wron…
The Better Angels of Machine Personality: How Personality Relates to LLM Safety
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Xi’an Jiaotong University（西安交通大学）；Renmin University of China（中国人民大学）；University of Chinese Academy of Sciences（中国科学院大学）
