恒等曲率拉普拉斯近似用于改进分布外检测
Identity Curvature Laplace Approximation for Improved Out-of-Distribution Detection
AI总结:
本文提出恒等曲率拉普拉斯近似(ICLA)方法,通过恒等曲率与先验精度优化改进分布外检测,在CIFAR-10、CIFAR-100和ImageNet上显著提升性能,并揭示特征嵌入与Fisher曲率对齐问题。
摘要:
不确定性估计在安全关键应用中至关重要,其中稳健的分布外(OOD)检测必不可少。传统的贝叶斯方法虽然有效,但往往受限于高昂的计算需求。作为替代方案,拉普拉斯近似提供了一种更实用且高效的不确定性估计方法。在本文中,我们引入了恒等曲率拉普拉斯近似(ICLA),这是一种新颖的方法,通过使用恒等曲率并优化先验精度,挑战了传统的后验协方差公式。这一创新设计显著提升了在CIFAR-10、CIFAR-100和ImageNet等知名数据集上的OOD检测性能,同时保持了校准分数。我们将这一改进归因于典型特征嵌入与Fisher信息矩阵所测量的曲率之间的对齐问题。我们的发现进一步得到支持,表明引入Fisher惩罚或锐度感知最小化技术可以大幅增强标准拉普拉斯近似的不确定性估计能力。
英文摘要:
Uncertainty estimation is crucial in safety-critical applications, where robust out-of-distribution (OOD) detection is essential. Traditional Bayesian methods, though effective, are often hindered by high computational demands. As an alternative, Laplace approximation offers a more practical and efficient approach to uncertainty estimation. In this paper, we introduce the Identity Curvature Laplace Approximation (ICLA), a novel method that challenges the conventional posterior covariance formulation by using identity curvature and optimizing prior precision. This innovative design significantly enhances OOD detection performance on well-known datasets such as CIFAR-10, CIFAR-100, and ImageNet, while maintaining calibration scores. We attribute this improvement to the alignment issues between typical feature embeddings and curvature as measured by the Fisher information matrix. Our findings are further supported by demonstrating that incorporating Fisher penalty or sharpness-aware minimization techniques can greatly enhance the uncertainty estimation capabilities of standard Laplace approximation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。