先定位再排序：基于知识的视觉问答中无训练实体识别的再探讨
Ground Then Rank: Revisiting Knowledge-Based VQA with Training-Free Entity Identification
AI总结：
针对知识型视觉问答中实体与证据双重定位瓶颈，提出解耦实体识别与段落排序的无训练IBA框架，利用MLLM候选名称选择与文本重排序器，在降低复杂度同时提升性能。
摘要：
基于知识的视觉问答（KB-VQA）需要将视觉查询与图像中可直接观察内容之外的外部知识进行关联。尽管近期多模态大语言模型（MLLM）展现出强大的感知能力，但在需要从细粒度实体和证据层面进行定位的KB-VQA任务上仍存在困难。现有的大多数多模态检索增强生成（MM-RAG）方法将实体判别和段落级证据排序紧密耦合到单个重排序阶段，导致成本高且泛化能力有限。本文从工作流角度重新审视现有MM-RAG解决方案，认为实体级和事实级定位都是关键瓶颈。我们观察到，尽管MLLM在开放式实体命名时常常失败，但当从少量候选名称中选择时，它们能更好地识别正确实体。基于这一发现，我们提出一个简单且无需训练的“先识别后回答”（IBA）框架，将实体识别与段落级重排序解耦。我们的方法提示MLLM仅使用候选名称选择高置信度实体，然后使用现成的文本重排序器进行证据选择。在Encyclopedic-VQA和InfoSeek上的实验表明，我们的方法在降低训练和推理复杂度的同时，持续优于微调的多模态重排序基线。额外分析显示，性能提升不仅来自更好的实体识别，还来自一旦正确实体确定后选择更具信息量的证据。我们的实现已公开以促进可复现性。
英文摘要：
Knowledge-Based Visual Question Answering (KB-VQA) requires grounding visual queries to external knowledge beyond directly observable content in images. While recent multi modal large language models (MLLMs) show strong perceptual abilities, they struggle on KB-VQA tasks requiring groundings from both fine-grained entity and evidence levels. Most existing multi-modal retrieval augmented generation (MM-RAG) methods tightly couple entity discrimination and section-level evidence ranking into a single re-ranking stage, leading to high cost and limited generalization. In this work, we revisit existing MM-RAG solutions from a workflow perspective and argue both entity-level and fact-level groundings are key bottlenecks. We observe that although MLLMs often fail under open-ended entity naming, they can better identify the correct entity when selecting from a small set of candidate names. Based on this insight, we propose a simple and training-free identify-before-answer IBA framework that decouples entity identification from section-level re-ranking. Our approach prompts an MLLM to select high-confidence entities using only candidate names, followed by an off-the-shelf textual re-ranker for evidence selection. Experiments on Encyclopedic-VQA and InfoSeek show that our method consistently outperforms fine-tuned multi-modal re-ranking baselines while reducing training and inference complexity. Additional analyses reveal that the improvements arise not only from better entity identification, but also from selecting more informative evidence once correct entity is fixed. Our implementation is made public to ease reproducibility.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
