公平性审计样本量计算简明教程
A Brief Tutorial on Sample Size Calculations for Fairness Audits
AI总结：
本教程针对公平性审计缺乏样本量指导及未考虑子群体样本不等的问题，给出确定所需子群体样本量的方法，以提升不公平检测假设检验的统计功效。
摘要：
在公平性审计中，一个标准目标是检测给定算法在不同子群体之间是否表现出显著差异。为此类审计的统计分析提供适当的统计功效，对于获得有信息量的公平性评估至关重要，因为这可确保在不公平存在时以高概率检测到它。然而，关于公平性审计所需数据量，现有指导有限，缺少与常用公平性指标相关、可直接应用的结果。此外，对子群体样本量不相等这一情况的考虑也有所缺失。在本教程中，我们通过说明如何确定所需的子群体样本量，以最大化用于检测不公平的假设检验的统计功效，来解决这些问题。我们的研究结果适用于二分类模型的审计，以及作为混淆矩阵汇总而导出的多种公平性指标。此外，我们还讨论了审计研究设计中能够提高审计结果可靠性的其他方面。
英文摘要：
In fairness audits, a standard objective is to detect whether a given algorithm performs substantially differently between subgroups. Properly powering the statistical analysis of such audits is crucial for obtaining informative fairness assessments, as it ensures a high probability of detecting unfairness when it exists. However, limited guidance is available on the amount of data necessary for a fairness audit, lacking directly applicable results concerning commonly used fairness metrics. Additionally, the consideration of unequal subgroup sample sizes is also missing. In this tutorial, we address these issues by providing guidance on how to determine the required subgroup sample sizes to maximize the statistical power of hypothesis tests for detecting unfairness. Our findings are applicable to audits of binary classification models and multiple fairness metrics derived as summaries of the confusion matrix. Furthermore, we discuss other aspects of audit study designs that can increase the reliability of audit results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
