面向视觉-语言模型的无词汇表少样本学习
Vocabulary-free few-shot learning for Vision-Language Models
AI总结：
本文针对视觉-语言模型在无法获得类别名称时的少样本分类问题，提出基于通用提示相似性得分的相似性映射（SiM）方法，无需手工设计提示词即可实现高效、可解释的分类，为无词汇表少样本学习建立了重要基线。
摘要：
视觉-语言模型（VLMs）在少样本适应方面的最新进展，极大地扩展了它们仅利用少量标注样本即可跨任务泛化的能力。然而，现有方法主要建立在利用精心设计的、任务特定的提示词来发挥这些模型强大的零样本先验能力之上。这种对预定义类别名称的依赖可能会限制其适用性，尤其是在无法获得或难以指定精确类别名称的场景中。为了解决这一局限，我们引入了面向VLMs的无词汇表少样本学习，这是一种目标类别实例（即图像）可用但其对应名称不可用的设置。我们提出了相似性映射（Similarity Mapping，SiM），这是一个简单而有效的基线方法，仅基于与一组通用提示（文本或视觉）的相似性得分来对目标实例进行分类，从而消除了对精心手工设计提示词的需求。尽管概念上简单直接，SiM表现出强大的性能，具有高计算效率（学习映射通常不到一秒钟），并通过将目标类别与通用提示关联起来提供可解释性。我们相信，我们的方法可以作为未来无词汇表少样本学习研究的重要基线。代码可在 https://github.com/MaxZanella/vocabulary-free-FSL 获取。
英文摘要：
Recent advances in few-shot adaptation for Vision-Language Models (VLMs) have greatly expanded their ability to generalize across tasks using only a few labeled examples. However, existing approaches primarily build upon the strong zero-shot priors of these models by leveraging carefully designed, task-specific prompts. This dependence on predefined class names can restrict their applicability, especially in scenarios where exact class names are unavailable or difficult to specify. To address this limitation, we introduce vocabulary-free few-shot learning for VLMs, a setting where target class instances - that is, images - are available but their corresponding names are not. We propose Similarity Mapping (SiM), a simple yet effective baseline that classifies target instances solely based on similarity scores with a set of generic prompts (textual or visual), eliminating the need for carefully handcrafted prompts. Although conceptually straightforward, SiM demonstrates strong performance, operates with high computational efficiency (learning the mapping typically takes less than one second), and provides interpretability by linking target classes to generic prompts. We believe that our approach could serve as an important baseline for future research in vocabulary-free few-shot learning. Code is available at https://github.com/MaxZanella/vocabulary-free-FSL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
