arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.03545cs.LGcs.NAmath.NA
收藏

基于回归的机器学习算法的无分布风险评估

Distribution-free risk assessment of regression-based machine learning algorithms

University of Oxford(牛津大学);Munich Re(慕尼黑再保险公司)

Sukrita Singh, Neeraj Sarna, Yuanyuan Li, Yang Li, Agni Orfanoudaki, Michael Berger

AI总结:

针对高风险场景下回归类机器学习模型的失效概率评估需求,提出基于共形预测的无分布风险评估方法,证明其结果具保守性,并通过多设置实验验证了方法准确性。

摘要:

近年来,机器学习算法的复杂度不断提升,且越来越多地被部署于实际应用场景。然而,在实际环境中使用机器学习技术时,尤其是在医疗、工程等高风险应用中,获取预测模型的失效概率至关重要。我们将这一问题称为风险评估任务。我们聚焦于回归算法,以及计算真实标签落在模型预测值周围所定义区间内的概率这一风险评估任务。我们采用共形预测方法解决该风险评估问题,该方法能够提供以给定概率包含真实标签的预测区间。利用这一覆盖特性,我们证明了我们得到的近似失效概率具有保守性,即其不低于该机器学习算法的真实失效概率。我们开展了大量实验,针对存在协变量偏移和不存在协变量偏移的问题,对所提方法的准确性进行了实证研究。我们的分析涵盖了不同的建模机制、数据集规模以及共形预测方法。

英文摘要:

Machine learning algorithms have grown in sophistication over the years and are increasingly deployed for real-life applications. However, when using machine learning techniques in practical settings, particularly in high-risk applications such as medicine and engineering, obtaining the failure probability of the predictive model is critical. We refer to this problem as the risk-assessment task. We focus on regression algorithms and the risk-assessment task of computing the probability of the true label lying inside an interval defined around the model's prediction. We solve the risk-assessment problem using the conformal prediction approach, which provides prediction intervals that are guaranteed to contain the true label with a given probability. Using this coverage property, we prove that our approximated failure probability is conservative in the sense that it is not lower than the true failure probability of the ML algorithm. We conduct extensive experiments to empirically study the accuracy of the proposed method for problems with and without covariate shift. Our analysis focuses on different modeling regimes, dataset sizes, and conformal prediction methodologies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑