arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.11604cs.LG
收藏

Neural Clamping:用于神经网络校准的联合输入扰动与温度缩放

Neural Clamping: Joint Input Perturbation and Temperature Scaling for Neural Network Calibration

The Chinese University of Hong Kong(香港中文大学);National Tsing Hua University(国立清华大学);IBM Research(IBM研究院)

Yung-Chen Tang, Pin-Yu Chen, Tsung-Yi Ho

AI总结:

本文提出后处理校准方法 Neural Clamping,通过联合可学习输入扰动与温度缩放变换预训练分类器,理论上优于温度缩放,并在多个数据集和模型上显著超越现有方法。

摘要:

神经网络校准是深度学习中确保模型预测置信度与真实正确性似然一致的关键任务。本文提出了一种名为 Neural Clamping 的后处理校准新方法,该方法通过可学习的通用输入扰动和输出温度缩放参数,对预训练分类器进行简单的联合输入输出变换。此外,我们提供了理论解释,说明为何 Neural Clamping 可证明优于温度缩放。在 BloodMNIST、CIFAR-100 和 ImageNet 图像识别数据集及多种深度神经网络模型上的评估表明,实验结果显示 Neural Clamping 显著优于最先进的后处理校准方法。代码可在 github.com/yungchentang/NCToolkit 获取,演示可在 huggingface.co/spaces/TrustSafeAI/NCTV 获取。

英文摘要:

Neural network calibration is an essential task in deep learning to ensure consistency between the confidence of model prediction and the true correctness likelihood. In this paper, we propose a new post-processing calibration method called Neural Clamping, which employs a simple joint input-output transformation on a pre-trained classifier via a learnable universal input perturbation and an output temperature scaling parameter. Moreover, we provide theoretical explanations on why Neural Clamping is provably better than temperature scaling. Evaluated on BloodMNIST, CIFAR-100, and ImageNet image recognition datasets and a variety of deep neural network models, our empirical results show that Neural Clamping significantly outperforms state-of-the-art post-processing calibration methods. The code is available at github.com/yungchentang/NCToolkit, and the demo is available at huggingface.co/spaces/TrustSafeAI/NCTV.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑