发表机构
University of Toronto; Georgia Institute of Technology(多伦多大学; 佐治亚理工学院)
机构由 AI 辅助整理,请以论文原文为准。AI 中文总结
本研究评估供应商控制早期预警系统中的事后公平性干预,发现干预仅重新分配而非减少差异,并可能加剧边缘群体负担,提出“公平性剧场”概念。
AI 中文摘要
公共机构越来越多地采购其设计无法检查或更改的AI系统。在高等教育中,专有的早期预警系统(EWS)使得学院除了调整模型输出以解决不平等问题之外几乎没有其他选择。这引发了一个问题:公平性工作如何在供应商、机构、顾问和学生之间协调,这些参与者改变这些系统的权力不平等?使用加拿大安大略省一所公立学院的学生记录,我们在模拟采购约束下对研究型EWS评估了六种事后公平性干预措施。我们比较了公平性、准确性和人口统计学差异,引入了错误类型剖析来追踪干预措施如何重新分配假阳性和假阴性。干预措施重新分配了差异,但没有一致地减少它们。两种实现有利于已经处于优势地位的群体,因为它们使用群体规模来定义劣势;小型边缘化群体仍然得不到良好服务。这些发现表明采购约束和实施选择如何塑造公平性工作的可能性。我们将由此产生的状况称为公平性剧场;仪表板指标趋同,而群体的错误负担持续存在或恶化。
英文摘要
Public institutions increasingly procure AI systems whose design they cannot inspect or change. In higher education, proprietary Early Warning Systems (EWS) leave colleges with few options beyond adjusting model outputs to address inequity. This raises the question of how fairness work is coordinated among vendors, institutions, advisors, and students with unequal power to change these systems? Using student records from a public college in Ontario, Canada, we evaluate six post-hoc fairness interventions on a research EWS under simulated procurement constraints. We compare fairness, accuracy, and demographic disparities, introducing error-type profiling to trace how interventions redistribute false positives and false negatives. Interventions redistributed disparities without consistently reducing them. Two implementations favored already-advantaged groups because they used group size to define disadvantage; small, marginalized groups remained poorly served. These findings show how procurement constraints and implementation choices shape the possibilities for fairness work. We call the resulting condition fairness theatre; dashboard metrics converge while groups' error burdens persist or worsen.