预测质量的评估
Assessment of the quality of a prediction
AI总结:
本文论证表观 Shannon 互信息是唯一合适的预测质量度量,并提出基于 Dirichlet 混合 skew-Student 分布的贝叶斯建模方法来估计其不确定性,以前列腺癌复发时间预测数据验证了方法的适用性。
摘要:
Shannon 定义了两个变量之间的互信息。我们说明了为什么一个变量与预测算法所做预测之间的真实互信息不是衡量预测质量的合适指标,而表观 Shannon 互信息(ASI)才是;事实上,它是唯一具有两组截然不同的理想性质清单的预测质量度量,这一点此前已由 de Finetti 及其他作者证明。然而,估计 ASI 的不确定性是一个困难的问题,因为 $j(x,y)=\log\frac{Q_y(x)}{P(x)}$ 的个体值分布具有长且非对称的重尾。我们提出了一种针对 $j(x,y)$ 分布的贝叶斯建模方法,从其後验分布中可以推断出 ASI 的不确定性。该方法基于 skew-Student 分布的 Dirichlet 混合。我们在一个用于预测前列腺癌复发时间的贝叶斯模型的数据上说明了其用途。我们相信,在无法推导出 $j(x,y)$ 样本的显式分布的大多数问题中,这种方法通常是适用的,尽管精确的建模参数可能需要调整以适应特定情况。
英文摘要:
Shannon defined the mutual information between two variables. We illustrate why the true mutual information between a variable and the predictions made by a prediction algorithm is not a suitable measure of prediction quality, but the apparent Shannon mutual information (ASI) is; indeed it is the unique prediction quality measure with either of two very different lists of desirable properties, as previously shown by de Finetti and other authors. However, estimating the uncertainty of the ASI is a difficult problem, because of long and non-symmetric heavy tails to the distribution of the individual values of $j(x,y)=\log\frac{Q_y(x)}{P(x)}$ We propose a Bayesian modelling method for the distribution of $j(x,y)$, from the posterior distribution of which the uncertainty in the ASI can be inferred. This method is based on Dirichlet-based mixtures of skew-Student distributions. We illustrate its use on data from a Bayesian model for prediction of the recurrence time of prostate cancer. We believe that this approach is generally appropriate for most problems, where it is infeasible to derive the explicit distribution of the samples of $j(x,y)$, though the precise modelling parameters may need adjustment to suit particular cases.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。