arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2212.05987cs.LG
收藏

使用鲁棒元学习方法的选择性分类

Selective classification using a robust meta-learning approach

Google Research India(谷歌研究院印度分院)

Nishant Jain, Karthikeyan Shanmugam, Pradeep Shenoy

AI总结:

提出一种基于双层优化的实例条件重加权辅助网络方法,通过最小化 dropout 方差作为元目标来捕获预测不确定性,在选择性分类和多个真实数据集上显著超越 SOTA。

摘要:

预测不确定性——即模型对自身在某个输入上准确率的自我认知——对于通过训练干预构建鲁棒模型以及测试时应用(如选择性分类)都至关重要。我们提出了一种新颖的实例条件重加权方法,该方法使用辅助网络捕获预测不确定性,并统一了这些训练时和测试时的应用。辅助网络在双层优化框架中使用元目标进行训练。我们提案的一个关键贡献是最小化 dropout 方差的元目标,这是对贝叶斯预测不确定性的近似。我们在对照实验中表明,通过该元目标能有效捕获各种特定的不确定性概念,而以往的方法只能捕获某些方面。这些结果转化为在现实环境中的显著收益——选择性分类、标签噪声、领域自适应、校准——以及跨数据集的收益:Imagenet、Cifar100、diabetic retinopathy、Camelyon、WILDs、Imagenet-C、-A、-R、Clothing1M 等。对于 Diabetic Retinopathy,我们在选择性分类中看到比 SOTA 高达 3.4%/3.3% 的准确率和 AUC 提升。我们还改进了诸如 PLEX 的大规模预训练模型。

英文摘要:

Predictive uncertainty-a model's self awareness regarding its accuracy on an input-is key for both building robust models via training interventions and for test-time applications such as selective classification. We propose a novel instance-conditioned reweighting approach that captures predictive uncertainty using an auxiliary network and unifies these train- and test-time applications. The auxiliary network is trained using a meta-objective in a bilevel optimization framework. A key contribution of our proposal is the meta-objective of minimizing the dropout variance, an approximation of Bayesian Predictive uncertainty. We show in controlled experiments that we effectively capture the diverse specific notions of uncertainty through this meta-objective, while previous approaches only capture certain aspects. These results translate to significant gains in real-world settings-selective classification, label noise, domain adaptation, calibration-and across datasets-Imagenet, Cifar100, diabetic retinopathy, Camelyon, WILDs, Imagenet-C,-A,-R, Clothing1M, etc. For Diabetic Retinopathy, we see upto 3.4%/3.3% accuracy and AUC gains over SOTA in selective classification. We also improve upon large-scale pretrained models such as PLEX.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑