连接点：通过代理推理实现无需训练的视觉接地
Connecting the Dots: Training-Free Visual Grounding via Agentic Reasoning
AI总结：
GroundingAgent通过代理推理实现无需微调的视觉接地，达到65.1%的零样本准确率，并在选择阶段实现90%的准确率，展示了LLM推理能力的重要性。
摘要：
视觉接地任务，即将文本查询与图像中的特定区域链接起来，在视觉-语言整合中起着关键作用。现有方法通常依赖于大量的任务特定注释和微调，限制了其在新或分布外场景中有效泛化的能力。为了解决这些限制，我们引入了GroundingAgent，一种新的代理视觉接地框架，该框架无需任何任务特定的微调。GroundingAgent采用一种结构化、迭代的推理机制，整合了预训练的开放词汇物体检测器、多模态大语言模型（MLLMs）和大语言模型（LLMs），通过联合语义和空间分析逐步细化候选区域。值得注意的是，GroundingAgent在广泛使用的基准（RefCOCO、RefCOCO+、RefCOCOg）上实现了平均零样本接地准确率为65.1%，完全无需微调。此外，通过将MLLM生成的描述替换为原始查询文本，仅在选择阶段的准确率就达到约90%，接近监督性能，凸显了LLM推理能力的重要性。GroundingAgent还提供了强大的可解释性，透明地展示了每一步推理过程，并提供了清晰的决策过程见解。
英文摘要：
Visual grounding, the task of linking textual queries to specific regions within images, plays a pivotal role in vision-language integration. Existing methods typically rely on extensive task-specific annotations and fine-tuning, limiting their ability to generalize effectively to novel or out-of-distribution scenarios. To address these limitations, we introduce GroundingAgent, a novel agentic visual grounding framework that operates without any task-specific fine-tuning. GroundingAgent employs a structured, iterative reasoning mechanism that integrates pretrained open-vocabulary object detectors, multimodal large language models (MLLMs), and large language models (LLMs) to progressively refine candidate regions through joint semantic and spatial analyses. Remarkably, GroundingAgent achieves an average zero-shot grounding accuracy of 65.1 % on widely-used benchmarks (RefCOCO, RefCOCO+, RefCOCOg), entirely without fine-tuning. Furthermore, by substituting MLLM-generated captions with the original query texts, the accuracy at the selection stage alone reaches approximately 90 %, closely matching supervised performance and underscoring the critical role of LLM reasoning capabilities. GroundingAgent also offers strong interpretability, transparently illustrating each reasoning step and providing clear insights into its decision-making process.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
