arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2010.11530stat.MLcs.LG
收藏

干预后的模型更新自相矛盾地引入了偏倚

Model updating after interventions paradoxically introduces bias

University of Edinburgh(爱丁堡大学);MRC Human Genetics Unit(医学研究委员会人类遗传学单元);The Alan Turing Institute(阿兰·图灵研究所);Durham University(杜伦大学);Kings College Hospital(国王学院医院);Wellcome Trust(惠康信托基金会);University of Warwick(华威大学);Warwick Mathematics Institute(华威数学研究所)

James Liley, Samuel R Emerson, Bilal A Mateen, Catalina A Vallejos, Louis J M Aslett, Sebastian J Vollmer

摘要:

机器学习正越来越多地用于生成预测模型,以用于从信用风险评估到临床决策支持的许多现实环境。最近的讨论强调了当现有的预测分数形成标准工作流程的一部分时,更新二元结果的预测分数的潜在问题,从而推动干预。在这种情况下,现有的分数诱导了一个额外的致病途径,导致当原始分数被替换时的误校准。我们提出了一个一般的因果框架来描述和解决这个问题,并证明了一个等价的制定部分观察到的马尔可夫决策过程。我们使用这个模型来证明这种“天真的更新”的影响时,反复执行。也就是说,我们表明,连续的预测分数可能会收敛到一个点,他们预测自己的效果,或可能最终趋向于两个值之间的稳定振荡,我们认为,这两个结果都是可取的。此外,我们证明,即使模型拟合程序改善,实际性能可能会恶化。我们补充这些研究结果与几个潜在的路线,以克服这些问题的讨论。

英文摘要:

Machine learning is increasingly being used to generate prediction models for use in a number of real-world settings, from credit risk assessment to clinical decision support. Recent discussions have highlighted potential problems in the updating of a predictive score for a binary outcome when an existing predictive score forms part of the standard workflow, driving interventions. In this setting, the existing score induces an additional causative pathway which leads to miscalibration when the original score is replaced. We propose a general causal framework to describe and address this problem, and demonstrate an equivalent formulation as a partially observed Markov decision process. We use this model to demonstrate the impact of such `naive updating' when performed repeatedly. Namely, we show that successive predictive scores may converge to a point where they predict their own effect, or may eventually tend toward a stable oscillation between two values, and we argue that neither outcome is desirable. Furthermore, we demonstrate that even if model-fitting procedures improve, actual performance may worsen. We complement these findings with a discussion of several potential routes to overcome these issues.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑