视觉-语言模型的半监督少样本适应
Semi-Supervised Few-Shot Adaptation of Vision-Language Models
AI总结：
本文提出一种半监督方法，通过传播文本引导的伪标签来提升视觉-语言模型在极低样本情况下的适应性能，减少标注工作量超过50%。
摘要：
视觉-语言模型（VLMs）在大规模、异质数据源上预训练，正变得越来越受欢迎，提供了丰富的多模态嵌入，能够高效地转移到新任务。一个特别相关的应用是少样本适应，其中只有少量标注示例可用，通过多模态线性探测来调整模型。在医学影像中，专门的VLMs在零样本和少样本图像分类中表现出有希望的性能，这对于减轻专家标注的高成本具有价值。然而，在极低样本情况下仍存在挑战：医学任务中固有的类别不平衡往往导致欠代表的类别，惩罚整体模型性能。为了解决这一限制，我们提出利用未标注数据，引入一个高效的半监督求解器，在少样本适应过程中传播文本引导的伪标签。所提出的方法使VLMs的适应变得更为经济，减少标注工作量在低样本情况下超过50%。
英文摘要：
Vision-language models (VLMs) pre-trained on large, heterogeneous data sources are becoming increasingly popular, providing rich multi-modal embeddings that enable efficient transfer to new tasks. A particularly relevant application is few-shot adaptation, where only a handful of annotated examples are available to adapt the model through multi-modal linear probes. In medical imaging, specialized VLMs have shown promising performance in zero- and few-shot image classification, which is valuable for mitigating the high cost of expert annotations. However, challenges remain in extremely low-shot regimes: the inherent class imbalances in medical tasks often lead to underrepresented categories, penalizing overall model performance. To address this limitation, we propose leveraging unlabeled data by introducing an efficient semi-supervised solver that propagates text-informed pseudo-labels during few-shot adaptation. The proposed method enables lower-budget annotation pipelines for adapting VLMs, reducing labeling effort by >50% in low-shot regimes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
