arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.02628cs.LGcs.AI
收藏

信任,但要验证:使用自我监督探测来提高可信度

Trust, but Verify: Using Self-Supervised Probing to Improve Trustworthiness

National University of Singapore(新加坡国立大学)

Ailin Deng, Shen Li, Miao Xiong, Zhirui Chen, Bryan Hooi

摘要:

可信的机器学习对于深度学习模型的实际部署至关重要。虽然最先进的模型在准确性方面取得了令人难以置信的良好性能,但最近的文献表明,不幸的是,它们的预测置信度得分不能被信任:例如,当做出错误的预测时,他们往往过于自信,甚至对明显的离群值也是如此。在本文中,我们介绍了一种新的自监督探测方法,它使我们能够检查和减轻训练模型的过度自信问题,从而提高其可信度。我们提供了一个简单而有效的框架,它可以灵活地应用到现有的信任相关的方法,即插即用的方式。在三个信任相关的任务(误分类检测,校准和分布外检测)在各种基准测试的广泛实验验证了我们提出的探测框架的有效性。

英文摘要:

Trustworthy machine learning is of primary importance to the practical deployment of deep learning models. While state-of-the-art models achieve astonishingly good performance in terms of accuracy, recent literature reveals that their predictive confidence scores unfortunately cannot be trusted: e.g., they are often overconfident when wrong predictions are made, or so even for obvious outliers. In this paper, we introduce a new approach of self-supervised probing, which enables us to check and mitigate the overconfidence issue for a trained model, thereby improving its trustworthiness. We provide a simple yet effective framework, which can be flexibly applied to existing trustworthiness-related methods in a plug-and-play manner. Extensive experiments on three trustworthiness-related tasks (misclassification detection, calibration and out-of-distribution detection) across various benchmarks verify the effectiveness of our proposed probing framework.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑