发表机构
University of Colorado Boulder（科罗拉多大学博尔德分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出基于狄利克雷-多项分布模型的贝叶斯方法，利用嵌套不确定性集结构高效评估模型校准，适用于昂贵或非分布输出的模拟，并能检测校准不良。
AI 中文摘要
我们提出一个贝叶斯框架，用于评估来自模拟研究的嵌套不确定性集的校准情况。校准通常通过从已知模型重复模拟数据，并将经验覆盖概率与其名义值进行比较来评估，这种方法通常需要数百或数千次独立的模型评估才能获得可靠的估计。当正向模型评估成本高昂时，例如在使用大型地球系统模型进行数据同化时，这种计算需求可能令人望而却步；或者当模拟的输出是单个或多个不确定性集而非完整分布时，这种方法可能不适用。我们提出一种方法，利用不确定性集在名义覆盖水平上的嵌套结构来评估这些模型的校准情况。该方法基于狄利克雷-多项分布模型，允许使用贝叶斯因子以高效方式检验模型校准的精确概念和基于区域的概念。与需要完整预测分布的方法不同，该方法适用于直接产生预测区间或置信区间的程序。我们在一个包含模型形式误差的数据同化问题上演示了该方法，并表明即使只有有限的模拟研究可用，它也能检测出校准不良。
英文摘要
We present a Bayesian framework for assessing the calibration of nested uncertainty sets from simulation studies. Calibration is typically evaluated by repeatedly simulating data from a known model and comparing empirical coverage probabilities with their nominal values, an approach that often requires hundreds or thousands of independent model evaluations to obtain reliable estimates. Such computational demands can be prohibitive when the forward models are expensive to evaluate, for example in data assimilation using a large Earth-system model, or may not be applicable, for exmaple when the output of the simulation is a single or multiple uncertainty sets rather than a full distribution. We present a method that leverages the nested structure of uncertainty sets across nominal coverage levels to assess the calibration of these models. The method, which is based on a Dirichlet-Multinomial model, allows for testing both exact and region-based notions of model calibration in an efficient manner using Bayes factors. Unlike methods that require a full predictive distribution, this method works for procedures that directly produce prediction or confidence intervals. We demonstrate the method on a data assimilation problem containing model form error and show that it is capable of detecting miscalibration even when there are only a limited number of simulation studies available.
