无需词汇的细粒度视觉识别 via 增强的上下文感知视觉语言模型
Vocabulary-free Fine-grained Visual Recognition via Enriched Contextually Grounded Vision-Language Model
AI总结：
本文提出E-FineR方法，无需词汇实现细粒度视觉识别，提供高可解释性和在零样本和少样本分类中的高性能。
摘要：
细粒度图像分类，即在更广泛类别内区分视觉相似的子类别（例如鸟类物种、汽车型号、花的类型），是具有挑战性的计算机视觉问题。传统方法依赖于固定词汇和封闭集分类范式，限制了其在现实世界中的可扩展性和适应性，因为在现实世界中经常出现新的类别。最近的研究表明，结合大型语言模型（LLMs）和视觉语言模型（VLMs）可以实现开放集识别，而无需预定义的类别标签。然而，现有方法在分类阶段往往难以充分利用LLMs的潜力，并且依赖于LLM提供的猜测类别名称，而没有经过彻底的分析和优化。为了解决这些瓶颈，我们提出我们的训练自由方法，即增强细粒度识别（或E-FineR，简称E-FineR），在细粒度视觉识别中实现了最先进的结果，同时提供了更高的可解释性，突显了其在现实世界和新领域中的强大潜力，特别是在专家标注难以获得的情况下。此外，我们展示了所提出方法在零样本和少样本分类中的应用，其中其性能与现有最先进的方法相当，且无需训练，也不需要人工干预。总体而言，我们的词汇自由框架支持了图像分类从刚性标签预测向灵活、语言驱动理解的转变，使可扩展且可推广的系统能够应用于现实世界。详细的代码可在https://github.com/demidovd98/e-finer上找到。
英文摘要：
Fine-grained image classification, the task of distinguishing between visually similar subcategories within a broader category (e.g., bird species, car models, flower types), is a challenging computer vision problem. Traditional approaches rely heavily on fixed vocabularies and closed-set classification paradigms, limiting their scalability and adaptability in real-world settings where novel classes frequently emerge. Recent research has demonstrated that combining large language models (LLMs) with vision-language models (VLMs) makes open-set recognition possible without the need for predefined class labels. However, the existing methods are often limited in harnessing the power of LLMs at the classification phase, and also rely heavily on the guessed class names provided by an LLM without thorough analysis and refinement. To address these bottlenecks, we propose our training-free method, Enriched-FineR (or E-FineR for short), which demonstrates state-of-the-art results in fine-grained visual recognition while also offering greater interpretability, highlighting its strong potential in real-world scenarios and new domains where expert annotations are difficult to obtain. Additionally, we demonstrate the application of our proposed approach to zero-shot and few-shot classification, where it demonstrated performance on par with the existing SOTA while being training-free and not requiring human interventions. Overall, our vocabulary-free framework supports the shift in image classification from rigid label prediction to flexible, language-driven understanding, enabling scalable and generalizable systems for real-world applications. Well-documented code is available on https://github.com/demidovd98/e-finer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
