arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.02634cs.LGcs.AI
收藏

基于共形预测的深度学习模型机载失校准检测

Onboard Out-of-Calibration Detection of Deep Learning Models using Conformal Prediction

German Aerospace Center (DLR)(德国航空航天中心)

Protim Bhattacharjee, Peter Jung

AI总结:

本文利用共形预测与模型不确定性的关系,提出一种检测深度学习模型是否失校准的方法,并在遥感数据集上验证了噪声场景下模型输出的不可信性。

摘要:

深度学习模型的黑箱特性使其在遥感等关键应用中的使用变得复杂。共形预测是一种在此类场景下确保可信度的方法。在数据可交换性的前提下,共形预测以预测集的形式提供有限样本覆盖保证,该预测集保证在用户定义的错误率内包含真实类别。在本文中,我们表明共形预测算法与深度学习模型的不确定性相关,并且这种关系可用于检测深度学习模型是否失校准。将Resnet50、Densenet161、InceptionV3和MobileNetV2等流行分类模型应用于EuroSAT等遥感数据集,以展示在噪声场景下模型输出如何变得不可信。此外,还提出了一种将模型不确定性与共形预测集平均大小相关联的失校准检测程序。

英文摘要:

The black box nature of deep learning models complicate their usage in critical applications such as remote sensing. Conformal prediction is a method to ensure trust in such scenarios. Subject to data exchangeability, conformal prediction provides finite sample coverage guarantees in the form of a prediction set that is guaranteed to contain the true class within a user defined error rate. In this letter we show that conformal prediction algorithms are related to the uncertainty of the deep learning model and that this relation can be used to detect if the deep learning model is out-of-calibration. Popular classification models like Resnet50, Densenet161, InceptionV3, and MobileNetV2 are applied on remote sensing datasets such as the EuroSAT to demonstrate how under noisy scenarios the model outputs become untrustworthy. Furthermore an out-of-calibration detection procedure relating the model uncertainty and the average size of the conformal prediction set is presented.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑