arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.09852cs.CLcs.AI
收藏

面向文本分布外检测的无监督逐层分数聚合

Unsupervised Layer-wise Score Aggregation for Textual OOD Detection

International Laboratory on Learning Systems(学习系统国际实验室);MILA - Quebec AI Institute(MILA - 魁北克人工智能研究所);McGill University(麦吉尔大学);CNRS(法国国家科学研究中心);INRIA(法国国家信息与自动化研究所);CEA(法国原子能和替代能源委员会)

Maxime Darrin, Guillaume Staerman, Eduardo Dadalto Câmara Gomes, Jackie CK Cheung, Pablo Piantanida, Pierre Colombo

AI总结:

本文针对文本OOD检测中最后一层嵌入并非最优的问题,提出一种数据驱动的无监督逐层异常分数聚合方法,并在扩展至最多77类的增强基准上取得接近预言机最佳层性能的稳健结果。

摘要:

分布外(OOD)检测是一个快速发展的领域,原因在于越来越多基于人工智能的系统带来了新的鲁棒性和安全性需求。现有的文本OOD检测器通常依赖在编码器最后一层嵌入输出上计算的异常分数(例如马氏距离)。在这项工作中,我们观察到OOD检测性能会因任务和层输出的不同而产生很大差异。更重要的是,我们表明通常的选择(最后一层)很少是OOD检测的最佳选择,如果能够挑选出最佳层,则可以取得远更好的结果。为了利用这一观察,我们提出了一种数据驱动的无监督方法来组合逐层异常分数。此外,我们扩展了经典的文本OOD基准,加入了类别数量更多(最多77类)的分类任务,从而反映更现实的设置。在这个增强基准上,我们表明所提出的后聚合方法在完全消除人工特征选择的同时,取得了稳健且一致的结果。其性能接近预言机(oracle)的最佳层性能。

英文摘要:

Out-of-distribution (OOD) detection is a rapidly growing field due to new robustness and security requirements driven by an increased number of AI-based systems. Existing OOD textual detectors often rely on an anomaly score (e.g., Mahalanobis distance) computed on the embedding output of the last layer of the encoder. In this work, we observe that OOD detection performance varies greatly depending on the task and layer output. More importantly, we show that the usual choice (the last layer) is rarely the best one for OOD detection and that far better results could be achieved if the best layer were picked. To leverage this observation, we propose a data-driven, unsupervised method to combine layer-wise anomaly scores. In addition, we extend classical textual OOD benchmarks by including classification tasks with a greater number of classes (up to 77), which reflects more realistic settings. On this augmented benchmark, we show that the proposed post-aggregation methods achieve robust and consistent results while removing manual feature selection altogether. Their performance achieves near oracle's best layer performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑