arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2206.09387cs.LG
收藏

面向分布外检测的对偶表示学习

Dual Representation Learning for Out-of-Distribution Detection

Macquarie University(麦考瑞大学)

Zhilin Zhao, Longbing Cao

AI总结:

针对深度神经网络易给分布外样本高置信度预测的问题,提出对偶表示学习方法,结合标签与分布判别表示实现分布外检测,性能优于现有最优方法。

摘要:

为对分布内样本进行分类,深度神经网络会依据信息瓶颈挖掘强标签相关信息,丢弃弱标签相关信息。来自与分布内样本不同分布的分布外样本,可能因获取到最少的强标签相关信息而被赋予出人意料的高置信度预测。为区分分布内与分布外样本,对偶表示学习(Dual Representation Learning,DRL)通过同时挖掘分布内样本的强标签与弱标签相关信息,让分布外样本更难获得高置信度预测。对于一个挖掘强标签相关信息以学习标签判别表示的预训练网络,DRL训练其辅助网络挖掘剩余的弱标签相关信息,以学习分布判别表示。具体而言,针对一个标签判别表示,DRL通过整合与该标签判别表示相似度较低的多样表示,构建其互补的分布判别表示。相应地,DRL结合标签判别表示与分布判别表示来检测分布外样本。实验表明,DRL在分布外检测任务上的表现优于当前最优方法。

英文摘要:

To classify in-distribution samples, deep neural networks explore strongly label-related information and discard weakly label-related information according to the information bottleneck. Out-of-distribution samples drawn from distributions differing from that of in-distribution samples could be assigned with unexpected high-confidence predictions because they could obtain minimum strongly label-related information. To distinguish in- and out-of-distribution samples, Dual Representation Learning (DRL) makes out-of-distribution samples harder to have high-confidence predictions by exploring both strongly and weakly label-related information from in-distribution samples. For a pretrained network exploring strongly label-related information to learn label-discriminative representations, DRL trains its auxiliary network exploring the remaining weakly label-related information to learn distribution-discriminative representations. Specifically, for a label-discriminative representation, DRL constructs its complementary distribution-discriminative representation by integrating diverse representations less similar to the label-discriminative representation. Accordingly, DRL combines label- and distribution-discriminative representations to detect out-of-distribution samples. Experiments show that DRL outperforms the state-of-the-art methods for out-of-distribution detection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑