arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.06217cs.CL
收藏

VI-OOD:面向文本分布外检测的统一表示学习框架

VI-OOD: A Unified Representation Learning Framework for Textual Out-of-distribution Detection

Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算机系)

Li-Ming Zhan, Bo Liu, Xiao-Ming Wu

AI总结:

针对文本 OOD 检测中条件似然最大化导致表示偏差的问题,本文提出基于联合似然和预训练 Transformers 的变分推断框架 VI-OOD,并在多类文本分类任务上验证其有效性。

摘要:

分布外(OOD)检测对于确保深度神经网络在各类应用中的安全性和可靠性具有关键作用。尽管视觉数据中的 OOD 检测受到越来越多关注,但文本 OOD 检测领域得到的关注仍然较少。只有少数研究尝试将通用 OOD 检测方法直接应用于自然语言处理(NLP)任务,而没有充分考虑文本数据的特性。本文利用 Transformers 深入研究文本 OOD 检测。我们首先指出现有 OOD 检测方法中普遍存在的一个关键问题:通过最大化条件似然 $p(y\mid x)$ 学到的有偏表示可能导致性能不佳。随后,我们提出一种新颖的 OOD 检测变分推断框架 VI-OOD,该框架最大化联合分布 $p(x,y)$ 的似然,而非 $p(y\mid x)$。VI-OOD 通过高效利用预训练 Transformers 的表示,专门针对文本 OOD 检测进行设计。在多种文本分类任务上的综合实验表明,VI-OOD 具有有效性和广泛适用性。我们的代码已发布于 \url{https://github.com/liam0949/LLM-OOD}。

英文摘要:

Out-of-distribution (OOD) detection plays a crucial role in ensuring the safety and reliability of deep neural networks in various applications. While there has been a growing focus on OOD detection in visual data, the field of textual OOD detection has received less attention. Only a few attempts have been made to directly apply general OOD detection methods to natural language processing (NLP) tasks, without adequately considering the characteristics of textual data. In this paper, we delve into textual OOD detection with Transformers. We first identify a key problem prevalent in existing OOD detection methods: the biased representation learned through the maximization of the conditional likelihood $p(y\mid x)$ can potentially result in subpar performance. We then propose a novel variational inference framework for OOD detection (VI-OOD), which maximizes the likelihood of the joint distribution $p(x, y)$ instead of $p(y\mid x)$. VI-OOD is tailored for textual OOD detection by efficiently exploiting the representations of pre-trained Transformers. Through comprehensive experiments on various text classification tasks, VI-OOD demonstrates its effectiveness and wide applicability. Our code has been released at \url{https://github.com/liam0949/LLM-OOD}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑