arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.03213cs.LGstat.ML
收藏

自举你的方差(Bootstrap Your Own Variance)

Bootstrap Your Own Variance

University of Göttingen(哥廷根大学);Apple(苹果公司)

Polina Turishcheva, Jason Ramapuram, Sinead Williamson, Dan Busbridge, Eeshan Dhekane, Russ Webb

AI总结:

本文提出 BYOV,将自监督 BYOL 与贝叶斯 BBB 结合,用于无标签不确定性估计,并在多种数据增强下取得更优校准与可靠性。

摘要:

理解模型不确定性对许多应用都很重要。我们提出 Bootstrap Your Own Variance(BYOV),将 Bootstrap Your Own Latent(BYOL)这一无负样本的自监督学习(SSL)算法,与用于估计模型后验的贝叶斯方法 Bayes by Backprop(BBB)相结合。我们发现,BYOV 相对于监督式 BBB 模型所学得的预测标准差能够被高斯分布很好地刻画,这为所学参数后验可用于无标签不确定性估计提供了初步证据。BYOV 改进了确定性 BYOL 基线(测试 ECE 提升 2.83%,测试 Brier 提升 1.03%),并在多种增强测试下表现出更好的校准与可靠性(例如:针对椒盐噪声,测试 ECE 提升 2.4%,测试 Brier 提升 1.2%)。

英文摘要:

Understanding model uncertainty is important for many applications. We propose Bootstrap Your Own Variance (BYOV), combining Bootstrap Your Own Latent (BYOL), a negative-free Self-Supervised Learning (SSL) algorithm, with Bayes by Backprop (BBB), a Bayesian method for estimating model posteriors. We find that the learned predictive std of BYOV vs. a supervised BBB model is well captured by a Gaussian distribution, providing preliminary evidence that the learned parameter posterior is useful for label free uncertainty estimation. BYOV improves upon the deterministic BYOL baseline (+2.83% test ECE, +1.03% test Brier) and presents better calibration and reliability when tested with various augmentations (eg: +2.4% test ECE, +1.2% test Brier for Salt & Pepper noise).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑