ESD：期望平方差作为一种无需调参的可训练校准度量
ESD: Expected Squared Difference as a Tuning-Free Trainable Calibration Measure
AI总结：
本文提出无超参数的可训练校准损失ESD，从两个期望的平方差刻画校准误差，并在CNN与Transformer等实验中证明其无需调参、校准更优且计算成本更低。
摘要：
研究表明，现代神经网络由于过度自信的预测，往往校准不佳。传统上，后处理方法被用于在训练后校准模型。近年来，研究者提出了各种可训练校准度量，将其直接纳入训练过程。然而，这些方法都包含内部超参数，且这些校准目标的性能依赖于调整这些超参数；随着神经网络和数据集规模变得更大，这会产生更多计算成本。因此，我们提出期望平方差（Expected Squared Difference，ESD），这是一种无需调参（即无超参数）的可训练校准目标损失，其中我们从两个期望之间平方差的角度来看待校准误差。通过在多种架构（CNN、Transformer）和数据集上进行大量实验，我们证明：（1）将ESD纳入训练可在各种批量大小设置下改善模型校准，而无需调整内部超参数；（2）与以往方法相比，ESD产生最佳校准结果；（3）由于不存在内部超参数，ESD大幅降低了训练期间校准所需的计算成本。代码可在https://github.com/hee-suk-yoon/ESD公开获取。
英文摘要：
Studies have shown that modern neural networks tend to be poorly calibrated due to over-confident predictions. Traditionally, post-processing methods have been used to calibrate the model after training. In recent years, various trainable calibration measures have been proposed to incorporate them directly into the training process. However, these methods all incorporate internal hyperparameters, and the performance of these calibration objectives relies on tuning these hyperparameters, incurring more computational costs as the size of neural networks and datasets become larger. As such, we present Expected Squared Difference (ESD), a tuning-free (i.e., hyperparameter-free) trainable calibration objective loss, where we view the calibration error from the perspective of the squared difference between the two expectations. With extensive experiments on several architectures (CNNs, Transformers) and datasets, we demonstrate that (1) incorporating ESD into the training improves model calibration in various batch size settings without the need for internal hyperparameter tuning, (2) ESD yields the best-calibrated results compared with previous approaches, and (3) ESD drastically improves the computational costs required for calibration during training due to the absence of internal hyperparameter. The code is publicly accessible at https://github.com/hee-suk-yoon/ESD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
