Out-of-distribution (OOD) detection plays a crucial role in ensuring the safety and reliability of deep neural networks in various applications. While there has been a growing focus on OOD detection in visual data, the field of textual OOD detection has received less attention. Only a few attempts have been made to directly apply general OOD detection methods to natural language processing (NLP) tasks, without adequately considering the characteristics of textual data. In this paper, we delve into textual OOD detection with Transformers. We first identify a key problem prevalent in existing OOD detection methods: the biased representation learned through the maximization of the conditional likelihood $p(y\mid x)$ can potentially result in subpar performance. We then propose a novel variational inference framework for OOD detection (VI-OOD), which maximizes the likelihood of the joint distribution $p(x, y)$ instead of $p(y\mid x)$. VI-OOD is tailored for textual OOD detection by efficiently exploiting the representations of pre-trained Transformers. Through comprehensive experiments on various text classification tasks, VI-OOD demonstrates its effectiveness and wide applicability. Our code has been released at \url{https://github.com/liam0949/LLM-OOD}.
TTL: Test-time Textual Learning for OOD Detection with Pretrained Vision-Language Models
Sun Yat-sen University(中山大学);China United Network Communications Corporation Limited Guangdong Branch(中国联合网络通信集团有限公司广东分公司);Peng Cheng Laboratory(鹏城实验室);Hong Kong University of Science and Technology(香港科技大学);Key Laboratory of Machine Intelligence and Advanced Computing, MOE(教育部机器智能与高级计算重点实验室)
Beyond Mahalanobis-Based Scores for Textual OOD Detection
Mathématiques et Informatique pour la Complexité et les Systèmes(复杂系统数学与计算机科学研究所);CentraleSupelec, Université Paris Saclay(巴黎萨克雷大学中央高等电子学院);Laboratoire des signaux et systemes(信号与系统实验室);CNRS, Université Paris-Saclay(巴黎萨克雷大学法国国家科学研究中心);Inria, CEA, Université Paris-Saclay(巴黎萨克雷大学法国国家信息与自动化研究所、法国原子能和替代能源委员会);International Laboratory on Learning Systems(国际学习系统实验室);McGill - ETS - MILA - CNRS - CentraleSupélec, Université Paris-Saclay(巴黎萨克雷大学麦吉尔大学-高等技术学院-蒙特利尔学习算法研究所-法国国家科学研究中心-中央高等电子学院)
Unsupervised Layer-wise Score Aggregation for Textual OOD Detection
International Laboratory on Learning Systems(学习系统国际实验室);MILA - Quebec AI Institute(MILA - 魁北克人工智能研究所);McGill University(麦吉尔大学);CNRS(法国国家科学研究中心);INRIA(法国国家信息与自动化研究所);CEA(法国原子能和替代能源委员会)
Disentangling Confidence Score Distribution for Out-of-Domain Intent Detection with Energy-Based Learning
Beijing University of Posts and Telecommunications(北京邮电大学);Meituan(美团)
在面向任务的对话系统中,从用户查询中检测出域外(OOD)或未知意图是必不可少的。传统的基于softmax的置信度得分容易受到过度自信问题的影响。在本文中,我们提出了一个简单但强大的基于能量的评分函数来检测OOD样本的能量分数高于IND样本。此外,给定一小组标记的OOD样本,我们引入了一个基于能量的利润目标监督OOD检测明确区分OOD样本从IND。综合实验和分析表明,该方法有助于理清IND和OOD数据的置信度分布。\footnote{Our code is available at \url{https://github.com/pris-nlp/EMNLP2022-energy_for_OOD/}.}