确定性模型中精确和校准不确定度的估计
Estimation of Accurate and Calibrated Uncertainties in Deterministic models
摘要:
在本文中,我们专注于问题的不确定性分配的单点预测所产生的确定性模型,输出一个连续变量。这个问题适用于任何最先进的物理或工程模型,其计算成本不容易允许运行集合并估计与单点预测相关的不确定性。从本质上讲,我们设计了一种方法,可以轻松地将确定性预测转换为概率性预测。我们表明,要做到这一点,人们必须在这样一个概率模型的准确性和可靠性(校准)之间做出妥协。因此,我们引入了一个成本函数来编码它们的权衡。我们使用连续秩概率得分来衡量准确性,我们推导出一个分析公式的可靠性,在连续标量变量的预测表示的高斯分布的情况下。新的精度可靠性成本函数,然后使用估计的输入相关的方差,给定一个黑盒均值函数,通过求解一个双目标优化问题。这一策略背后的简单理念是,基于估计方差的预测不仅应该准确,而且应该可靠(即与观测值在统计上一致)。相反,早期的作品的基础上最小化的经典成本函数,如负对数概率密度,不能同时执行准确性和可靠性。我们展示了几个例子,都与合成数据,其中潜在的隐藏噪声可以准确地恢复,并与大型现实世界的数据集。
英文摘要:
In this paper we focus on the problem of assigning uncertainties to single-point predictions generated by a deterministic model that outputs a continuous variable. This problem applies to any state-of-the-art physics or engineering models that have a computational cost that does not readily allow to run ensembles and to estimate the uncertainty associated to single-point predictions. Essentially, we devise a method to easily transform a deterministic prediction into a probabilistic one. We show that for doing so, one has to compromise between the accuracy and the reliability (calibration) of such a probabilistic model. Hence, we introduce a cost function that encodes their trade-off. We use the Continuous Rank Probability Score to measure accuracy and we derive an analytic formula for the reliability, in the case of forecasts of continuous scalar variables expressed in terms of Gaussian distributions. The new Accuracy-Reliability cost function is then used to estimate the input-dependent variance, given a black-box mean function, by solving a two-objective optimization problem. The simple philosophy behind this strategy is that predictions based on the estimated variances should not only be accurate, but also reliable (i.e. statistical consistent with observations). Conversely, early works based on the minimization of classical cost functions, such as the negative log probability density, cannot simultaneously enforce both accuracy and reliability. We show several examples both with synthetic data, where the underlying hidden noise can accurately be recovered, and with large real-world datasets.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。