发表机构
Johns Hopkins University（约翰·霍普金斯大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究如何评估机器学习辅助决策对下游结果的因果影响，提出利用先前RCT数据构建新模型因果效应界限的部分识别方法，通过两个单调性假设，经模拟研究表明该方法能产生更具信息量的界限。
AI 中文摘要
预测性机器学习模型越来越多地用于辅助医疗保健和刑事司法等高风险领域的人类决策者。人们越来越认识到需要评估部署这些系统对下游结果（如患者生存或犯罪再犯）的因果影响。随机对照试验（RCT）能提供有关已部署模型影响的高质量证据，但存在挑战：当模型更新或重新训练以提高预测性能时，通常无法进行重复试验。在这项工作中，我们提出一种使用先前RCT数据来构建新模型因果效应界限的部分识别方法。我们方法的核心创新是利用将细粒度预测准确性与下游结果相关联的假设。我们通过两个单调性假设来做到这一点：第一，关于个体层面的‘反事实正确性’（在其他条件相同的情况下，正确预测会导致非劣结果）；第二，关于子组预测性能与结果之间的关系，可解释为关于对模型输出信任的假设。我们通过模拟研究展示了我们的方法，说明与先前工作相比，纳入这些信息如何能产生更具信息量的界限。
英文摘要
Predictive machine learning (ML) models are increasingly used to aid human decision-makers across various high-risk domains such as healthcare and criminal justice. There is a growing recognition of the need to evaluate the causal impact of deploying these systems on downstream outcomes, such as patient survival or crime recidivism. Randomized control trials (RCTs) can provide high-quality evidence on the impact of a deployed model, but they run into a challenge: it is often infeasible to run repeated trials when models are updated or retrained to improve predictive performance. In this work, we present a partial-identification approach to using prior RCT data to construct bounds on the causal effect of a new model. The core innovation in our approach is to leverage assumptions relating fine-grained predictive accuracy to downstream outcomes. We do so via two monotonicity assumptions: first, on individual-level `counterfactual correctness' (all else being equal, a correct prediction leads to non-inferior outcomes); and second, on the relation between subgroup predictive performance and outcomes, interpretable as an assumption regarding trust in model outputs. We demonstrate our method with a simulation study, illustrating how incorporating this information can lead to more informative bounds compared to prior work.
Comments45 pages. In proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence
