arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.15019cs.LGcs.AIstat.ML
收藏

基于风格与内容信息的一致性引导温度缩放用于域外校准

Consistency-Guided Temperature Scaling Using Style and Content Information for Out-of-Domain Calibration

Korea Advanced Institute of Science and Technology (KAIST)(韩国科学技术院);Purdue University(普渡大学)

Wonjeong Choi, Jungwuk Park, Dong-Jun Han, Younghyun Park, Jaekyun Moon

AI总结:

本文提出一致性引导温度缩放(CTS),利用源域样本在风格和内容上的一致性相互监督来提升域外校准性能,在多个数据集上优于现有方法且不损失精度。

摘要:

近年来,关于深度神经网络对域偏移鲁棒性的研究兴趣迅速增长。然而,现有大多数工作侧重于提高模型精度,而非校准性能,而校准性能是可信人工智能系统的另一项重要要求。温度缩放(TS)是一种保持精度的后处理校准方法,已被证明在域内设置中有效,但在域外(OOD)设置中效果不佳,因为难以事先为未见域获得验证集。本文提出一致性引导温度缩放(CTS),这是一种新的温度缩放策略,通过在源域中的数据样本之间提供相互监督,能够显著提升OOD校准性能。我们观察到,不一致的样本预测所导致的过度自信是OOD校准的主要障碍,受此启发,我们提出从两个不同方面——风格和内容——考虑一致性来引导缩放过程,这两个方面是在多域设置中能够很好表示数据样本的关键组成部分。实验结果表明,我们提出的策略优于现有工作,在多个数据集上取得了优越的OOD校准性能。这可以通过仅使用源域来实现,且不牺牲精度,使我们的方案可直接应用于各种可信人工智能系统。

英文摘要:

Research interests in the robustness of deep neural networks against domain shifts have been rapidly increasing in recent years. Most existing works, however, focus on improving the accuracy of the model, not the calibration performance which is another important requirement for trustworthy AI systems. Temperature scaling (TS), an accuracy-preserving post-hoc calibration method, has been proven to be effective in in-domain settings, but not in out-of-domain (OOD) due to the difficulty in obtaining a validation set for the unseen domain beforehand. In this paper, we propose consistency-guided temperature scaling (CTS), a new temperature scaling strategy that can significantly enhance the OOD calibration performance by providing mutual supervision among data samples in the source domains. Motivated by our observation that over-confidence stemming from inconsistent sample predictions is the main obstacle to OOD calibration, we propose to guide the scaling process by taking consistencies into account in terms of two different aspects -- style and content -- which are the key components that can well-represent data samples in multi-domain settings. Experimental results demonstrate that our proposed strategy outperforms existing works, achieving superior OOD calibration performance on various datasets. This can be accomplished by employing only the source domains without compromising accuracy, making our scheme directly applicable to various trustworthy AI systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑