视觉语言模型的图像识别：基于视觉与语言嵌入
Image Recognition with Vision and Language Embeddings of VLMs
AI总结：
本研究评估了多种视觉语言模型的纯视觉与语言引导分类性能，提出基于每类精度的无学习融合方法，利用两者互补优势提升ImageNet-1k分类准确率。
摘要：
视觉语言模型（VLMs）通过图像-文本对齐实现了强大的零样本分类能力。然而，其纯视觉推理能力仍未得到充分探索。在本工作中，我们对多种双编码器视觉语言模型进行了全面的评估，涵盖语言引导和纯视觉图像分类，包括成熟模型和近期模型，如SigLIP 2和RADIOv2.5。我们在ImageNet-1k验证集及其标签修正版本上，以标准设置比较了性能。我们分析了影响准确性的关键因素，包括提示设计、类别多样性、k-NN中的邻居数量以及参考集大小。我们表明，语言和视觉提供了互补的优势，某些类别更偏好文本提示，而其他类别则通过视觉相似性处理得更好。为利用这种互补性，我们提出了一种简单、无需学习的融合方法，基于每类精度，从而提高了分类性能。代码可在以下网址获取：https://github.com/gonikisgo/bmvc2025-vlm-image-recognition。
英文摘要：
Vision-language models (VLMs) have enabled strong zero-shot classification through image-text alignment. Yet, their purely visual inference capabilities remain under-explored. In this work, we conduct a comprehensive evaluation of both language-guided and vision-only image classification with a diverse set of dual-encoder VLMs, including both well-established and recent models such as SigLIP 2 and RADIOv2.5. The performance is compared in a standard setup on the ImageNet-1k validation set and its label-corrected variant. The key factors affecting accuracy are analysed, including prompt design, class diversity, the number of neighbours in k-NN, and reference set size. We show that language and vision offer complementary strengths, with some classes favouring textual prompts and others better handled by visual similarity. To exploit this complementarity, we introduce a simple, learning-free fusion method based on per-class precision that improves classification performance. The code is available at: https://github.com/gonikisgo/bmvc2025-vlm-image-recognition.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
