发表机构
Swinburne University of Technology（斯威本科技大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
VFR-Audit是围绕裁决翻转率构建的框架，用于解决住院时长预测公平性审计的裁决不稳定性问题，可报告裁决翻转率及三类可靠性指标。
AI 中文摘要
临床人工智能中的公平性审计将连续型公平性指标转换为二元的通过/不通过裁决，以匹配操作阈值，医院管理委员会、支付方和监管机构会依据该裁决采取行动。这类审计会随时间及不同医院站点重复开展，因此同一裁决可能在不同审计中在通过与不通过间翻转。现有不确定性方法如贝叶斯后验、自助法置信区间和置换检验，仅在连续指标层面解决裁决不稳定性问题，将指标层面的不确定性转换为裁决稳定性声明仍是手动步骤，在审计覆盖的（模型、指标、属性）单元中扩展性差。现有不确定性方法还未明确，诸如重加权或按组阈值调整等偏差缓解步骤，是否会以AUROC或AUPRC衡量的模型判别力为代价，产生稳定的通过裁决。为解决这一裁决稳定性缺口，我们提出VFR-Audit，这是一个围绕裁决翻转率（Verdict Flip Rate，VFR）构建的框架，VFR是一个介于0和0.5之间的标量，用于衡量分层自助重采样下裁决反转的概率。VFR-Audit会报告VFR，以及三个可靠性维度：队列内重采样稳定性、审计规模敏感性，以及通过Fleiss' kappa衡量的跨医院裁决一致性。
英文摘要
Fairness audits in clinical Artificial Intelligence convert continuous fairness metrics into binary pass-or-fail verdicts against operational thresholds, where hospital governance boards, payers, and regulators act on the resulting verdicts. Such audits are repeated over time and across hospital sites, thus the same verdict can flip between pass and fail across audits. Existing uncertainty methods such as Bayesian posteriors, bootstrap confidence intervals, and permutation tests address verdict instability only at the continuous-metric level. Converting metric-level uncertainty into a verdict-stability claim remains a manual step that scales poorly across the (model, metric, attribute) cells an audit covers. Existing uncertainty methods also leave open whether bias-mitigation steps, such as reweighing or per-group threshold shifts, yield a stable passing verdict at the cost of model discrimination measured as AUROC or AUPRC.To address this verdict-stability gap, we propose VFR-Audit, a framework built around the Verdict Flip Rate (VFR), a scalar bounded between 0 and 0.5 that measures the probability of verdict reversal under stratified bootstrap resampling. VFR-Audit reports VFR alongside three reliability axes, namely within-cohort resampling stability, audit-size sensitivity, and cross-hospital verdict agreement via Fleiss' kappa.
Comments12 pages, 5 figures. Accepted at the 35th ACM International Conference on Information and Knowledge Management (CIKM 2026)
