不变性是否就是算法公平性所需的一切？移除人口统计信息可能产生新的偏见
Is invariance all you need for algorithmic fairness? Removing demographic information can create new bias
浏览论文内容
中文总结 AI 辅助
本文研究算法公平性中人口统计表示不变性的作用，证明强制不变性可能阻碍偏见缓解甚至产生新偏见，并区分边缘与类条件不变性，指出其分别对应人口统计均等和均等化几率，经理论和实证验证，认为不变性对公平性既不可取也不充分。
中文摘要 AI 辅助
模型内部表示中编码的人口统计信息通常被认为是算法偏见的风险因素，而人口统计表示的不变性常被吹捧为理想状态。然而，尽管人口统计捷径学习是一个真实的威胁，但当人口统计信息与目标标签相关时，一定程度的编码是必要的。在此，我们从数学和实证上证明，强制人口统计不变性实际上可能阻碍偏见缓解，甚至产生新的偏见。我们区分了边缘表示不变性和类条件表示不变性，并表明它们分别意味着标准的群体公平概念——人口统计均等和均等化几率。我们通过理论分析和在五个表格数据集及两个胸部X光影像数据集上的实证评估，检验了强制两种不变性类型对预测性能和公平性的影响。我们的研究结果支持我们的数学论证，即人口统计表示不变性对于公平性既不可取也不充分。
英文摘要
Encoded demographic information in internal model representations is a commonly assumed risk factor for algorithmic bias, with demographic representation invariance often being touted as the ideal state. However, while demographic shortcut learning is a genuine threat, some degree of encoding is necessary when demographics correlate with target labels. Here, we show, mathematically and empirically, that enforcing demographic invariance can actually hamper bias mitigation and even create new biases. We distinguish marginal from class-conditional representation invariance, and show that they imply the standard group fairness notions of demographic parity and equalized odds, respectively. We evaluate the effects on predictive performance and fairness of enforcing both invariance types, both theoretically and empirically across five tabular and two chest X-ray imaging datasets. Our findings support our mathematical argument that demographic representation invariance is neither desirable nor sufficient for fairness.
