Recent research suggests that predictions made by machine-learning models can amplify biases present in the training data. When a model amplifies bias, it makes certain predictions at a higher rate for some groups than expected based on training-data statistics. Mitigating such bias amplification requires a deep understanding of the mechanics in modern machine learning that give rise to that amplification. We perform the first systematic, controlled study into when and how bias amplification occurs. To enable this study, we design a simple image-classification problem in which we can tightly control (synthetic) biases. Our study of this problem reveals that the strength of bias amplification is correlated to measures such as model accuracy, model capacity, model overconfidence, and amount of training data. We also find that bias amplification can vary greatly during training. Finally, we find that bias amplification may depend on the difficulty of the classification task relative to the difficulty of recognizing group membership: bias amplification appears to occur primarily when it is easier to recognize group membership than class membership. Our results suggest best practices for training machine-learning models that we hope will help pave the way for the development of better mitigation strategies. Code can be found at https://github.com/facebookresearch/cv_bias_amplification.
How I Met Your Bias: Investigating Bias Amplification in Diffusion Models
LTCI, Télécom Paris, Institut Polytechnique de Paris（LTCI、 Télécom Paris、 Institut Polytechnique de Paris）；MaLGa-DIBRIS, University of Genova（MaLGa-DIBRIS、乌尔比诺大学）；AIGO, Istituto Italiano di Tecnologia（AIGO、意大利技术研究所）
Mitigating Gender Bias Amplification in Distribution by Posterior Regularization
Tsinghua University（清华大学）；University of California, Los Angeles（加州大学洛杉矶分校）
先进的机器学习技术提高了自然语言处理的性能。然而，最近的研究，例如，Zhao et al.（2017）表明，这些技术无意中捕获了隐藏在语料库中的社会偏见，并进一步放大了它。然而，他们的分析仅针对模型的最佳预测。在本文中，我们从分布的角度研究了性别偏见放大问题，并证明了偏见是放大的标签上的预测概率分布的观点。我们进一步提出了一种基于后验正则化的偏差缓解方法。在性能损失很小的情况下，我们的方法几乎可以消除分布中的偏置放大。我们的研究有助于理解偏差放大效应。
减轻机器学习系统中的偏见需要完善我们对偏见传播途径的理解：从社会结构到大规模数据，再到经过训练的模型对社会的影响。在这项工作中，我们专注于问题的一个方面，即偏差放大：模型放大其训练数据中存在的偏差的趋势。Zhao et al.（2017）在开创性的工作中引入了一种衡量偏差放大的指标。然而，正如我们所证明，该度量具有许多缺点，包括合并不同类型的偏置放大以及不能考虑受保护属性的变化的基本速率。我们介绍和分析了一种新的，去耦度量测量偏置放大，$\text{BiasAmp}_{\rightarrow}$（定向偏置放大）。我们彻底分析和讨论这一指标的技术假设和规范的影响。我们提供了有关其测量的建议，警告不要预测敏感属性，鼓励使用置信区间，由于在运行中的模型的公平性波动，并讨论了这个度量捕获的局限性。在本文中，我们的工作提供了一个疑问看技术测量的偏见放大，我们希望它包括我们的规范性思想的指导下。代码位于https://github.com/princetonvisualai/directional-bias-amp
Diffusing DeBias: Synthetic Bias Amplification for Model Debiasing
MaLGa-DIBRIS, University of Genoa, Italy（热那亚大学MaLGa-DIBRIS研究所）；AI for Good (AIGO), Istituto Italiano di Tecnologia, Genova, Italy（意大利技术研究院AI for Good（AIGO））；Telècom-Paris, Ecole Polytechnique Superior, France（巴黎电信高等学院）；Department of Computer Science, University of Verona, Italy（威尼斯大学计算机科学系）
When majority rules, minority loses: bias amplification of gradient descent
University of Lille（里尔大学）；Toulouse School of Economics（图卢兹经济学院）；Institut Universitaire de France (IUF)（法国大学研究院（IUF））；ANITI；Université de Toulouse（图卢兹大学）；INRIA（法国国家信息与自动化研究所）
Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMs
Faculty of Data Science, City University of Macau（数据科学学院，澳门城市大学）；Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences)（计算能力网络与信息安全重点实验室，教育部，山东计算机科学中心，齐鲁工业大学（山东科学院））；Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science（山东省计算能力互联网与服务计算重点实验室，山东省计算机科学基础研究中心）
A Systematic Bias of Machine Learning Regression Models and Its Correction: an Application to Imaging-based Brain Age Prediction
University of Maryland（马里兰大学）；School of Medicine（医学院）；Maryland Psychiatric Research Center（马里兰精神病研究中心）；The University of Maryland Institute for Health Computing (UM-IHC)（马里兰大学健康计算研究所）
