首页/论文/2012.15859

arXiv 2012.158592020-12cs.CL

Intrinsic Bias Metrics Do Not Correlate with Application Bias

University of Edinburgh（爱丁堡大学）；Rasa Technologies GmbH（Rasa技术公司）；Heriot-Watt University（赫瑞-瓦特大学）

Seraphina Goldfarb-Tarrant, Rebecca Marchant, Ricardo Muñoz Sanchez, Mugdha Pandya, Adam Lopez

备注：: In Proceedings of ACL 2021, 9 pages

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Natural Language Processing (NLP) systems learn harmful societal biases that cause them to amplify inequality as they are deployed in more and more situations. To guide efforts at debiasing these systems, the NLP community relies on a variety of metrics that quantify bias in models. Some of these metrics are intrinsic, measuring bias in word embedding spaces, and some are extrinsic, measuring bias in downstream tasks that the word embeddings enable. Do these intrinsic and extrinsic metrics correlate with each other? We compare intrinsic and extrinsic metrics across hundreds of trained models covering different tasks and experimental conditions. Our results show no reliable correlation between these metrics that holds in all scenarios across tasks and languages. We urge researchers working on debiasing to focus on extrinsic measures of bias, and to make using these measures more feasible via creation of new challenge sets and annotated test data. To aid this effort, we release code, a new intrinsic metric, and an annotated test set focused on gender bias in hate speech.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Toward Understanding Bias Correlations for Mitigation in NLP

University of Illinois Chicago（伊利诺伊大学芝加哥分校）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Arizona State University（亚利桑那州立大学）

Natural Language Processing (NLP) models have been found discriminative against groups of different social identities such as gender and race. With the negative consequences of these undesired biases, researchers have responded with unprecedented effort and proposed promising approaches for bias mitigation. In spite of considerable practical importance, current algorithmic fairness literature lacks an in-depth understanding of the relations between different forms of biases. Social bias is complex by nature. Numerous studies in social psychology identify the "generalized prejudice", i.e., gene…

2022-05-24cs.AIcs.CL

静态词嵌入内在与外在偏见指标在测量偏见对齐后的相关性分析

Analyzing Correlations Between Intrinsic and Extrinsic Bias Metrics of Static Word Embeddings With Their Measuring Biases Aligned

The University of Tokyo（东京大学）

本研究通过对齐内在与外在偏见指标的测量对象，分析静态词嵌入两类偏见指标的相关性，发现内在指标仅能预测特定场景下的NLP系统偏见行为。

2024-09-14cs.CL

How Far Can It Go?: On Intrinsic Gender Bias Mitigation for Text Classification

University of Antwerp（安特卫普大学）；KU Leuven（鲁汶大学）；Leuven.AI Institute for Artificial Intelligence（鲁汶人工智能研究所）；Weizenbaum Institute（魏泽鲍姆研究所）；TU Berlin（柏林工业大学）

To mitigate gender bias in contextualized language models, different intrinsic mitigation strategies have been proposed, alongside many bias metrics. Considering that the end use of these language models is for downstream tasks like text classification, it is important to understand how these intrinsic bias mitigation strategies actually translate to fairness in downstream tasks and the extent of this. In this work, we design a probe to investigate the effects that some of the major intrinsic gender bias mitigation strategies have on downstream text classification tasks. We discover that inste…

2023-01-30cs.CL

Comparing Intrinsic Gender Bias Evaluation Measures without using Human Annotated Examples

Tokyo Institute of Technology（东京工业大学）；University of Liverpool（利物浦大学）；Amazon（亚马逊）

Numerous types of social biases have been identified in pre-trained language models (PLMs), and various intrinsic bias evaluation measures have been proposed for quantifying those social biases. Prior works have relied on human annotated examples to compare existing intrinsic bias evaluation measures. However, this approach is not easily adaptable to different languages nor amenable to large scale evaluations due to the costs and difficulties when recruiting human annotators. To overcome this limitation, we propose a method to compare intrinsic gender bias evaluation measures without relying o…

2023-01-28cs.CL

偏见在基于编码器的视觉-语言模型中传播：从内在度量到零样本检索结果的系统分析

Biases Propagate in Encoder-based Vision-Language Models: A Systematic Analysis From Intrinsic Measures to Zero-shot Retrieval Outcomes

Carnegie Mellon University（卡内基梅隆大学）；Northwestern University（西北大学）；University of Washington（华盛顿大学）

本文提出受控框架，将VLM表征的内在社会群体偏见与零样本双向检索的外在偏见关联，发现二者高度相关且大模型传播更强。

2025-06-06cs.CL

How Gender Debiasing Affects Internal Model Representations, and Why It Matters

Technion – Israel Institute of Technology（以色列理工学院）；University of Edinburgh（爱丁堡大学）

Common studies of gender bias in NLP focus either on extrinsic bias measured by model performance on a downstream task or on intrinsic bias found in models' internal representations. However, the relationship between extrinsic and intrinsic bias is relatively unknown. In this work, we illuminate this relationship by measuring both quantities together: we debias a model during downstream fine-tuning, which reduces extrinsic bias, and measure the effect on intrinsic bias, which is operationalized as bias extractability with information-theoretic probing. Through experiments on two tasks and mult…

2022-05-16cs.CL

更多相关论文收起更多论文

缓解预训练语言模型中性别偏差的投影方法

Projective Methods for Mitigating Gender Bias in Pre-trained Language Models

National Research Council Canada（加拿大国家研究委员会）；University of Guelph（圭尔夫大学）

本研究探讨投影去偏方法在BERT内部表示中的应用，发现其能有效缓解内在及下游性别偏差，但两者相关性不强，提示不应仅依赖内在偏差测试集评估去偏效果。

2024-03-27cs.CL

Marked Attribute Bias in Natural Language Inference

University of Guelph（圭尔夫大学）；Vector Institute（向量研究所）

Reporting and providing test sets for harmful bias in NLP applications is essential for building a robust understanding of the current problem. We present a new observation of gender bias in a downstream NLP application: marked attribute bias in natural language inference. Bias in downstream applications can stem from training data, word embeddings, or be amplified by the model in use. However, focusing on biased word embeddings is potentially the most impactful first step due to their universal nature. Here we seek to understand how the intrinsic properties of word embeddings contribute to th…

2021-09-28cs.CL

CLIMB：大型语言模型中临床偏见的基准

CLIMB: A Benchmark of Clinical Bias in Large Language Models

USC（南加州大学）；UCLA（加利福尼亚大学洛杉矶分校）；UC Davis（加利福尼亚大学戴维斯分校）

本文提出CLIMB基准，通过新指标AssocMAD和反事实干预评估LLMs在临床决策中的内在与外在偏见，揭示Mistral和LLaMA模型存在的偏见问题。

2024-11-14cs.CL

从内在偏见与遗忘的视角理解任务无关型去偏

Towards Understanding Task-agnostic Debiasing Through the Lenses of Intrinsic Bias and Forgetfulness

Michigan State University（密歇根州立大学）；UC Santa Barbara（加州大学圣巴巴拉分校）

本文研究任务无关型去偏后模型在下游微调中重新学习偏见的问题，提出 ProSocialTuning 框架，通过正则化成功去偏的注意力头来克服遗忘问题，使微调模型接近偏见下界。

2024-06-06cs.CL

Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Tasks

Tokyo Institute of Technology（东京工业大学）；University of Liverpool（利物浦大学）；Amazon（亚马逊）

We study the relationship between task-agnostic intrinsic and task-specific extrinsic social bias evaluation measures for Masked Language Models (MLMs), and find that there exists only a weak correlation between these two types of evaluation measures. Moreover, we find that MLMs debiased using different methods still re-learn social biases during fine-tuning on downstream tasks. We identify the social biases in both training instances as well as their assigned labels as reasons for the discrepancy between intrinsic and extrinsic bias evaluation measurements. Overall, our findings highlight the…

2022-10-06cs.CL

在下游任务中应用内在去偏：机器翻译的挑战与考量

Applying Intrinsic Debiasing on Downstream Tasks: Challenges and Considerations for Machine Translation

Hebrew University of Jerusalem（耶路撒冷希伯来大学）；Allen Institute for AI（艾伦人工智能研究所）；University of Washington（华盛顿大学）

本研究系统测试了内在去偏方法对神经机器翻译模型的影响，揭示了去偏嵌入选择、单词与子词标记不匹配及目标语言差异三大挑战，并证明这些因素显著影响下游性能与去偏效果。

2024-06-02cs.CL

让我们在决策前忘却刻板印象：评估内在偏差缓解对大语言模型下游公平性的影响

Let's Unlearn Stereotypes Before Decision-Making: Assessing the Impact of Intrinsic Bias Mitigation on Downstream Fairness in LLMs

本研究提出公平感知概念忘却（FACU）方法，在保留预测性能的同时缓解LLM内在偏差，可提升下游公平性，结合反事实数据增强效果更佳，为LLM公平性优化提供了新路径。

2026-08-10cs.CLcs.CYcs.LG

面向语言模型零样本/少样本学习性能提升的基于提示的偏差校准

Prompt-Based Bias Calibration for Better Zero/Few-Shot Learning of Language Models

Purdue University（普渡大学）

针对预训练语言模型固有偏差导致提示式零/少样本学习性能不佳的问题，提出零输入提示的高效偏差校准方法，仅更新0.1%参数，在多数据集上显著提升两类场景的学习性能。

2024-10-06cs.CLcs.LG

上下文长度变化下孟加拉语偏见特征的实证研究

An Empirical Study on the Characteristics of Bias upon Context Length Variation for Bangla

Bangladesh University of Engineering and Technology (BUET)（孟加拉工程技术大学）

本文针对低资源语言孟加拉语，创建了性别偏见测量数据集，调整了现有偏见测量方法，并实证揭示了上下文长度变化对偏见指标的显著影响，为孟加拉语偏见分析提供了基础资源。

2024-06-25cs.CL

理解大型语言模型中的内在社会经济偏见

Understanding Intrinsic Socioeconomic Biases in Large Language Models

HEC Montreal（蒙特利尔高等商学院）；Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；McGill University（麦吉尔大学）

本研究通过百万级英文句子数据集，系统量化了LLMs中的人口统计与社会经济偏见，发现交叉性显著放大偏见，并强调需主动缓解以保障公平决策。

2024-05-28cs.CLcs.CYcs.LG