基于预训练Transformer的伪离群点暴露方法用于分布外检测
Pseudo Outlier Exposure for Out-of-Distribution Detection using Pretrained Transformers
AI总结：
针对预训练语言模型难检测语义相近分布外样本且需额外收集离群数据的问题，提出伪离群点暴露方法，通过掩码分布内类别相关词元构建替代离群数据集，无需外部离群数据即可训练拒绝网络，在多个文本分类基准上性能具备竞争力。
摘要：
对于现实世界的语言应用而言，检测分布外（OOD）样本有助于提醒用户或拒绝这类不可靠样本。然而，现代过参数化语言模型往往会对分布内（ID）和OOD样本都产生过度自信的预测。尤其是，语言模型难以应对与ID样本具有相似语义表示的OOD样本，因为这些OOD样本位于ID流形附近。可以用ID样本和多样化的离群点样本训练拒绝网络来检测测试阶段的OOD样本，但显式收集辅助OOD数据集会带来额外的数据收集负担。本文提出了一种简单而有效的方法，名为伪离群点暴露（POE），该方法通过依次掩码与ID类别相关的词元来构建替代OOD数据集。POE引入的替代OOD样本呈现出与ID数据相似的表示，这在训练拒绝网络时最为有效。我们的方法不需要任何外部OOD数据，并且可以在现成的Transformer中轻松实现。与最先进算法的全面比较表明，POE在多个文本分类基准上具有竞争力。
英文摘要：
For real-world language applications, detecting an out-of-distribution (OOD) sample is helpful to alert users or reject such unreliable samples. However, modern over-parameterized language models often produce overconfident predictions for both in-distribution (ID) and OOD samples. In particular, language models suffer from OOD samples with a similar semantic representation to ID samples since these OOD samples lie near the ID manifold. A rejection network can be trained with ID and diverse outlier samples to detect test OOD samples, but explicitly collecting auxiliary OOD datasets brings an additional burden for data collection. In this paper, we propose a simple but effective method called Pseudo Outlier Exposure (POE) that constructs a surrogate OOD dataset by sequentially masking tokens related to ID classes. The surrogate OOD sample introduced by POE shows a similar representation to ID data, which is most effective in training a rejection network. Our method does not require any external OOD data and can be easily implemented within off-the-shelf Transformers. A comprehensive comparison with state-of-the-art algorithms demonstrates POE's competitiveness on several text classification benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
