arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.00675cs.CVcs.AI
收藏

LLM与视觉-语言模型结合实现零样本单类分类

LLM meets Vision-Language Models for Zero-Shot One-Class Classification

IMT Atlantique(IMT大西洋理工大学);Sony Europe, B.V.(索尼欧洲有限公司);Mila(米拉计算科学研究所)

Yassir Bendou, Giulia Lioi, Bastien Pasdeloup, Lukas Mauch, Ghouthi Boukli Hacene, Fabien Cardinaux, Vincent Gripon

AI总结:

提出两步法,利用LLM生成混淆物体列表,结合CLIP实现仅凭标签的零样本单类分类,并在新基准上超越现有方法。

摘要:

我们考虑零样本单类视觉分类问题,将传统单类分类扩展到仅目标类标签可用的场景。该方法旨在无需目标类示例的情况下,区分正负查询样本。我们提出两步解决方案:首先查询大型语言模型以获取视觉上易混淆的物体,然后依赖视觉-语言预训练模型(如CLIP)进行分类。通过改编大规模视觉基准,我们展示了所提方法在此设置下优于改编的现成替代方案。具体而言,我们提出了一个现实基准,其中负查询样本与正样本来自同一原始数据集,包括iNaturalist的粒度控制版本,其中负样本在分类树中与正样本保持固定距离。据我们所知,我们是首个仅利用标签就能区分单个类别与其他语义相关类别的演示。

英文摘要:

We consider the problem of zero-shot one-class visual classification, extending traditional one-class classification to scenarios where only the label of the target class is available. This method aims to discriminate between positive and negative query samples without requiring examples from the target class. We propose a two-step solution that first queries large language models for visually confusing objects and then relies on vision-language pre-trained models (e.g., CLIP) to perform classification. By adapting large-scale vision benchmarks, we demonstrate the ability of the proposed method to outperform adapted off-the-shelf alternatives in this setting. Namely, we propose a realistic benchmark where negative query samples are drawn from the same original dataset as positive ones, including a granularity-controlled version of iNaturalist, where negative samples are at a fixed distance in the taxonomy tree from the positive ones. To our knowledge, we are the first to demonstrate the ability to discriminate a single category from other semantically related ones using only its label.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑