将视觉-语言模型从标志性适应到包容性：用于无标签的多标签识别
Adapting Vision-Language Models from Iconic to Inclusive for Multi-Label Recognition Without Labels
AI总结：
提出无监督框架，通过“切割”和“缝合”两阶段适应VLMs，实现无标签的多标签图像识别，在四个数据集上超越现有无监督方法。
摘要：
理解多标签图像仍然是计算机视觉中的一项挑战性任务。随着视觉-语言多模态学习的快速发展，视觉-语言模型（VLM）能够在没有标注数据的情况下实现零样本识别。然而，由于其内在设计，这些模型通常优先考虑最标志性的物体，而忽略其他上下文正例。这种内在偏差与多标签学习的性质相冲突，从而限制了它们的适用性。在这项工作中，我们提出了一个无监督框架，将VLM从标志性识别适应到包容性理解，实现无标签的多标签图像识别。我们的方法包括两个关键阶段：“切割”和“缝合”：在切割阶段，我们提出了多采样响应估计器，以防止模型仅关注单个物体。在第二个缝合阶段，引入了多目标混合适应，以调整标签使其更符合多标签分布，同时仅在一个epoch内保留原始模型的内在特性。大量实验表明，我们的框架在四个公共数据集上显著优于现有的无监督方法，甚至超过了几种有代表性的弱监督基线。这些结果证明了将预训练VLM适应于更全面的视觉理解而无需人工标注的潜力。我们的代码在此https URL公开。
英文摘要：
Understanding multi-label images remains a challenging task in computer vision. With the rapid progress of vision-language multimodal learning, vision-language models (VLMs) enable zero-shot recognition without labeled data. However, due to their intrinsic design, these models often prioritize the most iconic object and omit other contextual positives. This intrinsic bias conflicts with the nature of multi-label learning, thereby limiting their applicability. In this work, we propose an unsupervised framework that adapts VLMs from iconic recognition toward inclusive understanding, enabling label-free multi-label image recognition. Our approach consists of two key stages, ``cutting'' and ``sewing'': In the cutting stage, we present the multi-sampling response estimator to prevent the model from concentrating only on one single object. In the second sewing stage, the multi-object blend adaptation is introduced to adjust the labels to better conform to the multi-label distribution while preserving the intrinsic characteristics of the original model within only one epoch. Extensive experiments show that our framework significantly outperforms existing unsupervised approaches on four public datasets, even surpassing several representative weakly supervised baselines. These results demonstrate the potential of adapting pre-trained VLMs for more comprehensive visual understanding without manual annotations. Our code is publicly available at https://github.com/iCVTEAM/TailorCLIP.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
