减轻机器学习系统中的偏见需要完善我们对偏见传播途径的理解:从社会结构到大规模数据,再到经过训练的模型对社会的影响。在这项工作中,我们专注于问题的一个方面,即偏差放大:模型放大其训练数据中存在的偏差的趋势。Zhao et al.(2017)在开创性的工作中引入了一种衡量偏差放大的指标。然而,正如我们所证明,该度量具有许多缺点,包括合并不同类型的偏置放大以及不能考虑受保护属性的变化的基本速率。我们介绍和分析了一种新的,去耦度量测量偏置放大,$\text{BiasAmp}_{\rightarrow}$(定向偏置放大)。我们彻底分析和讨论这一指标的技术假设和规范的影响。我们提供了有关其测量的建议,警告不要预测敏感属性,鼓励使用置信区间,由于在运行中的模型的公平性波动,并讨论了这个度量捕获的局限性。在本文中,我们的工作提供了一个疑问看技术测量的偏见放大,我们希望它包括我们的规范性思想的指导下。代码位于https://github.com/princetonvisualai/directional-bias-amp
英文摘要:
Mitigating bias in machine learning systems requires refining our understanding of bias propagation pathways: from societal structures to large-scale data to trained models to impact on society. In this work, we focus on one aspect of the problem, namely bias amplification: the tendency of models to amplify the biases present in the data they are trained on. A metric for measuring bias amplification was introduced in the seminal work by Zhao et al. (2017); however, as we demonstrate, this metric suffers from a number of shortcomings including conflating different types of bias amplification and failing to account for varying base rates of protected attributes. We introduce and analyze a new, decoupled metric for measuring bias amplification, $\text{BiasAmp}_{\rightarrow}$ (Directional Bias Amplification). We thoroughly analyze and discuss both the technical assumptions and normative implications of this metric. We provide suggestions about its measurement by cautioning against predicting sensitive attributes, encouraging the use of confidence intervals due to fluctuations in the fairness of models across runs, and discussing the limitations of what this metric captures. Throughout this paper, we work to provide an interrogative look at the technical measurement of bias amplification, guided by our normative ideas of what we want it to encompass. Code is located at https://github.com/princetonvisualai/directional-bias-amp
How I Met Your Bias: Investigating Bias Amplification in Diffusion Models
LTCI, Télécom Paris, Institut Polytechnique de Paris(LTCI、 Télécom Paris、 Institut Polytechnique de Paris);MaLGa-DIBRIS, University of Genova(MaLGa-DIBRIS、乌尔比诺大学);AIGO, Istituto Italiano di Tecnologia(AIGO、意大利技术研究所)
Mitigating Gender Bias Amplification in Distribution by Posterior Regularization
Tsinghua University(清华大学);University of California, Los Angeles(加州大学洛杉矶分校)
先进的机器学习技术提高了自然语言处理的性能。然而,最近的研究,例如,Zhao et al.(2017)表明,这些技术无意中捕获了隐藏在语料库中的社会偏见,并进一步放大了它。然而,他们的分析仅针对模型的最佳预测。在本文中,我们从分布的角度研究了性别偏见放大问题,并证明了偏见是放大的标签上的预测概率分布的观点。我们进一步提出了一种基于后验正则化的偏差缓解方法。在性能损失很小的情况下,我们的方法几乎可以消除分布中的偏置放大。我们的研究有助于理解偏差放大效应。
When majority rules, minority loses: bias amplification of gradient descent
University of Lille(里尔大学);Toulouse School of Economics(图卢兹经济学院);Institut Universitaire de France (IUF)(法国大学研究院(IUF));ANITI;Université de Toulouse(图卢兹大学);INRIA(法国国家信息与自动化研究所)
Diffusing DeBias: Synthetic Bias Amplification for Model Debiasing
MaLGa-DIBRIS, University of Genoa, Italy(热那亚大学MaLGa-DIBRIS研究所);AI for Good (AIGO), Istituto Italiano di Tecnologia, Genova, Italy(意大利技术研究院AI for Good(AIGO));Telècom-Paris, Ecole Polytechnique Superior, France(巴黎电信高等学院);Department of Computer Science, University of Verona, Italy(威尼斯大学计算机科学系)
Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMs
Faculty of Data Science, City University of Macau(数据科学学院,澳门城市大学);Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences)(计算能力网络与信息安全重点实验室,教育部,山东计算机科学中心,齐鲁工业大学(山东科学院));Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science(山东省计算能力互联网与服务计算重点实验室,山东省计算机科学基础研究中心)