IIR-VLM：面向大视觉-语言模型的上下文实例识别
IIR-VLM: In-Context Instance-level Recognition for Large Vision-Language Models
AI总结：
IIR-VLM通过整合预训练的ILR专家模型，提升大视觉-语言模型在上下文实例识别中的性能，有效解决细粒度辨别问题。
摘要：
实例级识别（ILR）关注区分个体实例，以人再识别为例。尽管现代VLMs在视觉感知方面表现出色，但我们发现其在ILR上的表现令人不满，通常大幅劣于领域特定的ILR模型。这一限制阻碍了许多VLMs的实用应用，例如在识别熟悉的人和物体对有效视觉理解至关重要的情况下。现有解决方案通常通过实例特定的数据集学习识别个体，这不仅导致显著的数据收集和训练成本，而且在细粒度辨别上也遇到困难。在本工作中，我们提出了IIR-VLM，一种增强的VLM，用于上下文实例级识别。我们整合了预训练的ILR专家模型作为辅助视觉编码器，以提供专门的特征用于学习多样化的实例，这使VLMs能够以单次学习的方式在上下文中学习新的实例。进一步，IIR-VLM利用这一知识进行实例感知的视觉理解。我们验证了IIR-VLM在现有实例个性化基准上的有效性。最后，我们在一个具有挑战性的新基准上展示了其优越的ILR性能，该基准评估了在不同难度和多样类别下的ILR能力，实例包括人、面孔、宠物和一般物体。
英文摘要：
Instance-level recognition (ILR) concerns distinguishing individual instances from one another, with person re-identification as a prominent example. Despite the impressive visual perception capabilities of modern VLMs, we find their performance on ILR unsatisfactory, often dramatically underperforming domain-specific ILR models. This limitation hinders many practical application of VLMs, e.g. where recognizing familiar people and objects is crucial for effective visual understanding. Existing solutions typically learn to recognize instances one at a time using instance-specific datasets, which not only incur substantial data collection and training costs but also struggle with fine-grained discrimination. In this work, we propose IIR-VLM, a VLM enhanced for In-context Instance-level Recognition. We integrate pre-trained ILR expert models as auxiliary visual encoders to provide specialized features for learning diverse instances, which enables VLMs to learn new instances in-context in a one-shot manner. Further, IIR-VLM leverages this knowledge for instance-aware visual understanding. We validate IIR-VLM's efficacy on existing instance personalization benchmarks. Finally, we demonstrate its superior ILR performance on a challenging new benchmark, which assesses ILR capabilities across varying difficulty and diverse categories, with person, face, pet and general objects as the instances at task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
