随机种子对临床分类器公平性的影响
On the Impact of Random Seeds on the Fairness of Clinical Classifiers
摘要：
最近的研究表明，微调大型网络对随机种子的变化非常敏感。我们探讨了这种现象对MIMIC-III中电子健康记录（EHR）临床预测任务中人口统计学群体模型公平性的影响-MIMIC-III是临床NLP研究的标准数据集。表观亚组性能差异很大的种子，产生类似的整体性能，虽然没有证据表明整体和亚组性能之间的权衡。然而，我们也发现，小样本量固有的少数群体的交叉点和有点罕见的条件限制了我们准确估计差距的能力。此外，我们发现，联合优化高的整体性能和低差距不会产生统计上显着的改善。我们的研究结果表明，使用MIMIC-III的公平工作应仔细考虑随机性和小样本量可能产生的明显差异的变化。
英文摘要：
Recent work has shown that fine-tuning large networks is surprisingly sensitive to changes in random seed(s). We explore the implications of this phenomenon for model fairness across demographic groups in clinical prediction tasks over electronic health records (EHR) in MIMIC-III -- the standard dataset in clinical NLP research. Apparent subgroup performance varies substantially for seeds that yield similar overall performance, although there is no evidence of a trade-off between overall and subgroup performance. However, we also find that the small sample sizes inherent to looking at intersections of minority groups and somewhat rare conditions limit our ability to accurately estimate disparities. Further, we find that jointly optimizing for high overall performance and low disparities does not yield statistically significant improvements. Our results suggest that fairness work using MIMIC-III should carefully account for variations in apparent differences that may arise from stochasticity and small sample sizes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
