arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2306.01293cs.CV
收藏

LoCoOp:基于提示学习的少样本分布外检测

LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learning

The University of Tokyo(东京大学);LY Corporation;Tokyo University of Science(东京理科大学)

Atsuyuki Miyai, Qing Yu, Go Irie, Kiyoharu Aizawa

AI总结:

本文提出了一种名为 LoCoOp 的新型视觉-语言提示学习方法,通过利用 CLIP 局部特征作为分布外特征进行正则化,有效解决了少样本分布外检测中文本嵌入存在无关信息的问题,并在大规模基准上超越了零样本和全监督方法。

摘要:

我们提出了一种用于少样本分布外(OOD)检测的新型视觉-语言提示学习方法。少样本 OOD 检测旨在仅使用少量标记的分布内(ID)图像,检测出训练期间未见过的类别的 OOD 图像。尽管 CoOp 等提示学习方法在少样本 ID 分类中表现出有效性和高效性,但由于文本嵌入中可能存在与 ID 无关的信息,它们在 OOD 检测中仍面临局限。为解决此问题,我们引入了一种名为局部正则化上下文优化(LoCoOp)的新方法,该方法在训练期间利用 CLIP 局部特征的部分作为 OOD 特征来执行 OOD 正则化。CLIP 的局部特征包含大量与 ID 无关的干扰因素(如背景),通过学习将这些干扰因素推离 ID 类别文本嵌入,我们可以消除 ID 类别文本嵌入中的干扰因素,并增强 ID 和 OOD 之间的分离。在大规模 ImageNet OOD 检测基准上的实验证明了我们的 LoCoOp 优于零样本、全监督检测方法和提示学习方法。值得注意的是,即使在单样本设置下——每个类别仅有一个标签,LoCoOp 也优于现有的零样本和全监督检测方法。代码将在 https://github.com/AtsuMiyai/LoCoOp 上提供。

英文摘要:

We present a novel vision-language prompt learning approach for few-shot out-of-distribution (OOD) detection. Few-shot OOD detection aims to detect OOD images from classes that are unseen during training using only a few labeled in-distribution (ID) images. While prompt learning methods such as CoOp have shown effectiveness and efficiency in few-shot ID classification, they still face limitations in OOD detection due to the potential presence of ID-irrelevant information in text embeddings. To address this issue, we introduce a new approach called Local regularized Context Optimization (LoCoOp), which performs OOD regularization that utilizes the portions of CLIP local features as OOD features during training. CLIP's local features have a lot of ID-irrelevant nuisances (e.g., backgrounds), and by learning to push them away from the ID class text embeddings, we can remove the nuisances in the ID class text embeddings and enhance the separation between ID and OOD. Experiments on the large-scale ImageNet OOD detection benchmarks demonstrate the superiority of our LoCoOp over zero-shot, fully supervised detection methods and prompt learning methods. Notably, even in a one-shot setting -- just one label per class, LoCoOp outperforms existing zero-shot and fully supervised detection methods. The code will be available via https://github.com/AtsuMiyai/LoCoOp.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑