使用KCal后退一步：深度神经网络的基于多类内核的校准
Taking a Step Back with KCal: Multi-Class Kernel-Based Calibration for Deep Neural Networks
摘要：
深度神经网络（DNN）分类器通常过于自信，产生错误校准的类概率。在医疗保健等高风险应用中，从业人员需要$\textit{fully calibrated}$概率预测来进行决策。也就是说，以预测$\textit{vector}$为条件，$\textit{every}$类的概率应该接近预测值。大多数现有的校准方法要么缺乏产生校准输出的理论保证，降低分类精度的过程中，或只校准预测类。本文提出了一种新的基于核的校准方法KCal。与现有的校准程序不同，KCal不直接对DNN的logits或softmax输出进行操作。相反，KCal在倒数第二层潜在嵌入上学习度量空间，并使用校准集上的核密度估计生成预测。我们首先从理论上分析KCal，表明它享有可证明的$\textit{full}$校准保证。然后，通过在各种数据集上进行广泛的实验，我们表明KCal始终优于基线，这是通过校准误差和适当的评分规则（如Brier Score）来衡量的。
英文摘要：
Deep neural network (DNN) classifiers are often overconfident, producing miscalibrated class probabilities. In high-risk applications like healthcare, practitioners require $\textit{fully calibrated}$ probability predictions for decision-making. That is, conditioned on the prediction $\textit{vector}$, $\textit{every}$ class' probability should be close to the predicted value. Most existing calibration methods either lack theoretical guarantees for producing calibrated outputs, reduce classification accuracy in the process, or only calibrate the predicted class. This paper proposes a new Kernel-based calibration method called KCal. Unlike existing calibration procedures, KCal does not operate directly on the logits or softmax outputs of the DNN. Instead, KCal learns a metric space on the penultimate-layer latent embedding and generates predictions using kernel density estimates on a calibration set. We first analyze KCal theoretically, showing that it enjoys a provable $\textit{full}$ calibration guarantee. Then, through extensive experiments across a variety of datasets, we show that KCal consistently outperforms baselines as measured by the calibration error and by proper scoring rules like the Brier Score.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
