回归诊断遇上预测评估：条件校准、可靠性图与决定系数
Regression Diagnostics meets Forecast Evaluation: Conditional Calibration, Reliability Diagrams, and Coefficient of Determination
AI总结：
本文提出基于概率论的校准框架，适用于预测分布和点预测，引入T可靠性图并定义通用决定系数R*，以统一评估模型诊断与预测性能。
摘要：
模型诊断和预测评估是一枚硬币的两面。一个共同的原则是，拟合或预测的分布应当是校准的或可靠的，理想情况下是在自校准的意义上，即结果是来自假定分布的随机抽取。对于二元响应，这是可靠性的普遍概念。对于实值结果，尽管最近对分布回归和机器学习的兴趣激增，但一般的校准理论一直难以捉摸。我们开发了一个植根于概率论的框架，它产生了校准的层次结构，并适用于预测分布和独立的点预测。简而言之，一个预测——无论是分布的还是单值的——如果在泛函 T 方面可以按面值接受，那么它就是条件 T 校准的。每当 T 通过识别函数定义时——如阈值（非）超越概率、分位数、期望分位数和矩的情况——自校准就意味着 T 校准。我们引入了 T 可靠性图的总体版本，并重新审视了将得分分解为误校准（MCB）、区分度（DSC）和不确定性（UNC）的度量。在经验设置中，T 可靠性图和得分分量的稳定高效估计量通过非参数保序回归和池相邻违例者算法产生。对于样本内模型诊断，我们提出了一个通用的决定系数，$$\text{R}^\ast = \frac{\text{DSC}-\text{MCB}}{\text{UNC}},$$ 它在最小二乘（均值）回归中嵌套并重新解释了经典的 $\text{R}^2$ 及其在分位数回归中的自然类比 $\text{R}^1$，且适用于一般的 T 回归，在适度条件下具有 MCB $\geq 0$，DSC $\geq 0$，和 $\text{R}^\ast \in [0,1]$。
英文摘要：
Model diagnostics and forecast evaluation are two sides of the same coin. A common principle is that fitted or predicted distributions ought to be calibrated or reliable, ideally in the sense of auto-calibration, where the outcome is a random draw from the posited distribution. For binary responses, this is the universal concept of reliability. For real-valued outcomes, a general theory of calibration has been elusive, despite a recent surge of interest in distributional regression and machine learning. We develop a framework rooted in probability theory, which gives rise to hierarchies of calibration, and applies to both predictive distributions and stand-alone point forecasts. In a nutshell, a prediction - distributional or single-valued - is conditionally T-calibrated if it can be taken at face value in terms of the functional T. Whenever T is defined via an identification function - as in the cases of threshold (non) exceedance probabilities, quantiles, expectiles, and moments - auto-calibration implies T-calibration. We introduce population versions of T-reliability diagrams and revisit a score decomposition into measures of miscalibration (MCB), discrimination (DSC), and uncertainty (UNC). In empirical settings, stable and efficient estimators of T-reliability diagrams and score components arise via nonparametric isotonic regression and the pool-adjacent-violators algorithm. For in-sample model diagnostics, we propose a universal coefficient of determination, $$\text{R}^\ast = \frac{\text{DSC}-\text{MCB}}{\text{UNC}},$$ that nests and reinterprets the classical $\text{R}^2$ in least squares (mean) regression and its natural analogue $\text{R}^1$ in quantile regression, yet applies to T-regression in general, with MCB $\geq 0$, DSC $\geq 0$, and $\text{R}^\ast \in [0,1]$ under modest conditions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
