简单性偏差导致性能差异被放大
Simplicity Bias Leads to Amplified Performance Disparities
AI总结：
本文揭示模型对简单样本的偏好会造成与人口统计群体对齐的难度差异，并证明常用模型选择标准会放大这种差异，仅平衡群体样本量不足以保证性能公平。
摘要：
给定模型会觉得数据集的哪些部分困难？近期研究表明，经 SGD 训练的模型存在对简单性的偏差，这使它们优先学习多数类，或依赖有害的虚假相关性。在本文中，我们表明对“容易”内容的偏好要深刻得多：模型可能会优先处理数据集中它认为简单的任何类或群体，而牺牲它认为复杂的内容，这一点可由测试集上的性能差异来衡量。当具有不同复杂度水平的子集与人口统计群体相对应时，我们将其称为难度差异；即使在缺乏群体/标签关联的平衡数据集中，这种现象也会发生。我们展示了难度差异如何成为一个依赖于模型的量，并且在通过典型平均性能分数选择的常用模型中会被进一步放大。我们量化了一系列设置下的放大因子，以便在固定数据集上比较不同模型的差异。最后，我们给出两个现实世界中难度放大实际发生的例子，结果表明即使使用平衡数据集，群体之间也会出现比预期更糟的性能差异。平衡数据集中此类差异的存在表明，仅仅平衡各群体的样本量并不足以确保无偏的性能。我们希望这项工作为可度量地理解模型偏差与数据结构相互作用时所起的作用迈出一步，并呼吁在数据集审计之外，同时部署更多依赖于模型的缓解方法。
英文摘要：
Which parts of a dataset will a given model find difficult? Recent work has shown that SGD-trained models have a bias towards simplicity, leading them to prioritize learning a majority class, or to rely upon harmful spurious correlations. Here, we show that the preference for "easy" runs far deeper: A model may prioritize any class or group of the dataset that it finds simple-at the expense of what it finds complex-as measured by performance difference on the test set. When subsets with different levels of complexity align with demographic groups, we term this difficulty disparity, a phenomenon that occurs even with balanced datasets that lack group/label associations. We show how difficulty disparity is a model-dependent quantity, and is further amplified in commonly-used models as selected by typical average performance scores. We quantify an amplification factor across a range of settings in order to compare disparity of different models on a fixed dataset. Finally, we present two real-world examples of difficulty amplification in action, resulting in worse-than-expected performance disparities between groups even when using a balanced dataset. The existence of such disparities in balanced datasets demonstrates that merely balancing sample sizes of groups is not sufficient to ensure unbiased performance. We hope this work presents a step towards measurable understanding of the role of model bias as it interacts with the structure of data, and call for additional model-dependent mitigation methods to be deployed alongside dataset audits.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
