神经网络校准度量的解耦
Decoupling of neural network calibration measures
AI总结:
本研究探讨神经网络校准度量间的耦合,指出ECE等度量在确定最优校准时的不足,并提出AUSE作为残差不确定性的间接度量,以支持自动驾驶安全关键功能的认证。
摘要:
当前,大量工作致力于保障自动驾驶系统的安全性,这些系统严重依赖用于计算机视觉的深度神经网络。我们研究了不同神经网络校准度量之间的耦合关系,特别关注稀疏化误差曲线下面积(AUSE)这一度量。我们详细阐述了使用期望校准误差(ECE)确定最优校准时已知的不一致性问题,并证明了类似问题也存在于AUSE、不确定性校准分数(UCS)以及不确定性校准误差(UCE)中。我们得出结论,当前的方法论存在一个自由度,这阻碍了安全关键功能认证中唯一模型校准的实现。此外,我们提出将AUSE作为残差不确定性的间接度量,这种不确定性对于固定网络架构是不可约的,它由底层数据生成过程中的随机性(偶然贡献)以及假设空间的局限性(认知贡献)共同驱动。
英文摘要:
A lot of effort is currently invested in safeguarding autonomous driving systems, which heavily rely on deep neural networks for computer vision. We investigate the coupling of different neural network calibration measures with a special focus on the Area Under the Sparsification Error curve (AUSE) metric. We elaborate on the well-known inconsistency in determining optimal calibration using the Expected Calibration Error (ECE) and we demonstrate similar issues for the AUSE, the Uncertainty Calibration Score (UCS), as well as the Uncertainty Calibration Error (UCE). We conclude that the current methodologies leave a degree of freedom, which prevents a unique model calibration for the homologation of safety-critical functionalities. Furthermore, we propose the AUSE as an indirect measure for the residual uncertainty, which is irreducible for a fixed network architecture and is driven by the stochasticity in the underlying data generation process (aleatoric contribution) as well as the limitation in the hypothesis space (epistemic contribution).
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。