arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.06374stat.MLcs.LGstat.ME
收藏

用于安全预测模型更新的留存集

Holdouts set for safe predictive model updating

Durham University(杜伦大学);Alan Turing Institute(阿兰·图灵研究所)

Sami Haidar-Wehbe, Samuel R Emerson, Louis J M Aslett, James Liley

AI总结:

提出使用“留存集”进行安全预测模型更新,以避免直接更新导致的风险估计偏差,并证明该方法能将不良结局频率降至渐近最优水平,同时给出最优留存集大小的估计算法。

摘要:

用于不良结局的预测风险评分在指导健康干预中日益重要。由于所建模的分布发生变化,这些评分可能需要定期更新。然而,直接更新用于指导干预的风险评分可能导致有偏的风险估计。为解决这一问题,我们提出使用“留存集”进行更新——即人群中不接收由风险评分指导干预的一个子集。平衡留存集的大小至关重要,以确保更新后的风险评分具有良好的性能,同时最小化被留出的样本数量。我们证明该方法将不良结局频率降低至渐近最优水平,并指出通常不存在有竞争力的替代方案。我们描述了可以容易地识别最优留存集大小(OHS)的条件,并引入了用于OHS估计的参数化和半参数化算法。我们将这些方法应用于子痫前期的ASPRE风险评分,为在底层数据分布发生变化时更新该评分推荐了一项计划。我们表明,为了最小化随时间推移的子痫前期病例数,最好使用约10,000名个体的留存集来实现这一目标。

英文摘要:

Predictive risk scores for adverse outcomes are increasingly crucial in guiding health interventions. Such scores may need to be periodically updated due to change in the distributions they model. However, directly updating risk scores used to guide intervention can lead to biased risk estimates. To address this, we propose updating using a `holdout set' - a subset of the population that does not receive interventions guided by the risk score. Balancing the holdout set size is essential to ensure good performance of the updated risk score whilst minimising the number of held out samples. We prove that this approach reduces adverse outcome frequency to an asymptotically optimal level and argue that often there is no competitive alternative. We describe conditions under which an optimal holdout size (OHS) can be readily identified, and introduce parametric and semi-parametric algorithms for OHS estimation. We apply our methods to the ASPRE risk score for pre-eclampsia to recommend a plan for updating it in the presence of change in the underlying data distribution. We show that, in order to minimise the number of pre-eclampsia cases over time, this is best achieved using a holdout set of around 10,000 individuals.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑