arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.04072cs.CVcs.LG
收藏

基于标签传播的视觉-语言模型零样本分类

Label Propagation for Zero-shot Classification with Vision-Language Models

Czech Technical University in Prague(布拉格捷克技术大学);NAVER LABS Europe(NAVER欧洲实验室)

Vladan Stojnić, Yannis Kalantidis, Giorgos Tolias

AI总结:

针对含无标签数据的零样本分类,提出 ZLaP,利用图文特征图上的测地距离标签传播及高效归纳推理,在14个数据集上超过最新方法。

摘要:

视觉-语言模型(VLM)在零样本分类(即仅提供类别名称列表时进行的分类)上已展现出令人瞩目的性能。本文研究在存在无标签数据情况下的零样本分类问题。我们利用无标签数据的图结构,提出 ZLaP,这是一种基于标签传播(LP)并利用测地距离进行分类的方法。我们将 LP 定制为适用于同时包含文本和图像特征的图,并进一步提出一种基于对偶解和稀疏化步骤的高效归纳推理方法。我们在 14 个常用数据集上进行了大量实验以评估该方法的有效性,结果表明 ZLaP 优于最新的相关工作。代码:https://github.com/vladan-stojnic/ZLaP

英文摘要:

Vision-Language Models (VLMs) have demonstrated impressive performance on zero-shot classification, i.e. classification when provided merely with a list of class names. In this paper, we tackle the case of zero-shot classification in the presence of unlabeled data. We leverage the graph structure of the unlabeled data and introduce ZLaP, a method based on label propagation (LP) that utilizes geodesic distances for classification. We tailor LP to graphs containing both text and image features and further propose an efficient method for performing inductive inference based on a dual solution and a sparsification step. We perform extensive experiments to evaluate the effectiveness of our method on 14 common datasets and show that ZLaP outperforms the latest related works. Code: https://github.com/vladan-stojnic/ZLaP

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑