将置信度校准扩展到一般化的变异度量
Extending confidence calibration to generalised measures of variation
AI总结：
本文提出VCE度量，用于评估机器学习模型对任意变异度量的校准性，并通过实验展示其优于现有熵基度量的特性。
摘要：
我们提出了变异性校准误差（VCE）度量，用于评估机器学习分类器的校准性。该度量可视为已知的期望校准误差（ECE）的扩展，后者评估最大概率或置信度的校准性。其他衡量概率分布变异性的方法也存在，例如香农熵，其优势在于考虑整个概率分布。我们展示了如何将ECE方法从评估置信度校准扩展到评估任何变异度量的校准性。我们通过合成预测进行数值示例，这些预测经过设计完美校准，证明在该场景下，VCE具有期望的性质，即随着数据样本数的增加，VCE接近零，而另一种基于熵的校准度量（UCE）在文献中已被提出。
英文摘要：
We propose the Variation Calibration Error (VCE) metric for assessing the calibration of machine learning classifiers. The metric can be viewed as an extension of the well-known Expected Calibration Error (ECE) which assesses the calibration of the maximum probability or confidence. Other ways of measuring the variation of a probability distribution exist which have the advantage of taking into account the full probability distribution, for example the Shannon entropy. We show how the ECE approach can be extended from assessing confidence calibration to assessing the calibration of any metric of variation. We present numerical examples upon synthetic predictions which are perfectly calibrated by design, demonstrating that, in this scenario, the VCE has the desired property of approaching zero as the number of data samples increases, in contrast to another entropy-based calibration metric (the UCE) which has been proposed in the literature.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
