arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.17263cs.LGcs.CYstat.ML
收藏

偏差放大的有效理论

An Effective Theory of Bias Amplification

UCLA(加利福尼亚大学洛杉矶分校);Meta FAIR;Concordia University(康考迪亚大学);Mila(米拉人工智能研究所)

Arjun Subramonian, Samuel J. Bell, Levent Sagun, Elvis Dohmatob

AI总结:

本文针对机器学习中的偏差放大问题,在带或不带随机投影的岭回归框架下建立精确分析理论,解释模型与数据因素导致的群体性能差异,并经合成与半合成实验验证。

摘要:

机器学习模型能够捕捉并放大数据中存在的偏差,从而导致不同社会群体之间的测试性能差异。为了更好地理解、评估并缓解这些偏差,需要更深入地从理论上认识模型设计选择和数据分布属性如何促成偏差。在本工作中,我们在岭回归(包括使用和不使用随机投影)的情境下提出了一套精确的分析理论,其中前者在一种简化机制中对前馈神经网络进行建模。我们的理论为机器学习偏差提供了统一且严谨的解释,并对不同特征和参数机制中的偏差放大、少数群体偏差等现象给出洞见。例如,我们观察到,可能存在一个最优正则化惩罚或训练时间以避免偏差放大,而且不同群体之间可能存在测试误差差异,这些差异并不会因参数化程度提高而得到缓解。重要的是,我们的理论预测与机器学习偏差文献中报告的实证观察结果一致。我们在合成数据集和半合成数据集上对该理论进行了广泛的实证验证。

英文摘要:

Machine learning models can capture and amplify biases present in data, leading to disparate test performance across social groups. To better understand, evaluate, and mitigate these biases, a deeper theoretical understanding of how model design choices and data distribution properties contribute to bias is needed. In this work, we contribute a precise analytical theory in the context of ridge regression, both with and without random projections, where the former models feedforward neural networks in a simplified regime. Our theory offers a unified and rigorous explanation of machine learning bias, providing insights into phenomena such as bias amplification and minority-group bias in various feature and parameter regimes. For example, we observe that there may be an optimal regularization penalty or training time to avoid bias amplification, and there can be differences in test error between groups that are not alleviated with increased parameterization. Importantly, our theoretical predictions align with empirical observations reported in the literature on machine learning bias. We extensively empirically validate our theory on synthetic and semi-synthetic datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑