公平性的脆弱性：面向公平机器学习的因果敏感性分析
The Fragility of Fairness: Causal Sensitivity Analysis for Fair Machine Learning
AI总结：
针对现实数据偏差导致公平性评估失效的问题，提出基于因果敏感性分析的通用框架，在14个数据集上验证了公平性评估对微小偏差的脆弱性，为衡量指标信息量提供必要工具。
摘要：
公平性指标是公平机器学习文献（FairML）中的核心工具，用于判定ML模型在某种意义上是“公平”的。然而，现实世界的数据通常受到各种测量偏差和其他被违背假设的困扰，这可能使公平性评估失去意义。我们将因果敏感性分析的工具适配到FairML语境中，提供了一个通用框架，该框架（1）容纳了在“忽略设置”下可提出的几乎任何公平性指标与偏差的组合；（2）允许研究人员调查偏差的组合，从而产生非线性敏感性；（3）能够灵活编码特定领域的约束和假设。利用该框架，我们在14个经典公平性数据集上分析了3种分类器下最常见的奇偶性指标的敏感性。我们的分析揭示了公平性评估对即使是微小的数据集偏差也表现出惊人的脆弱性。我们表明，因果敏感性分析为衡量奇偶性指标评估的信息量提供了一个强大且必要的工具包。我们的代码库可在此获取：https://github.com/Jakefawkes/fragile_fair.
英文摘要：
Fairness metrics are a core tool in the fair machine learning literature (FairML), used to determine that ML models are, in some sense, "fair". Real-world data, however, are typically plagued by various measurement biases and other violated assumptions, which can render fairness assessments meaningless. We adapt tools from causal sensitivity analysis to the FairML context, providing a general framework which (1) accommodates effectively any combination of fairness metric and bias that can be posed in the "oblivious setting"; (2) allows researchers to investigate combinations of biases, resulting in non-linear sensitivity; and (3) enables flexible encoding of domain-specific constraints and assumptions. Employing this framework, we analyze the sensitivity of the most common parity metrics under 3 varieties of classifier across 14 canonical fairness datasets. Our analysis reveals the striking fragility of fairness assessments to even minor dataset biases. We show that causal sensitivity analysis provides a powerful and necessary toolkit for gauging the informativeness of parity metric evaluations. Our repository is available here: https://github.com/Jakefawkes/fragile_fair.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
