arXiv 2606.161582026-06-16cs.CVcs.CL

必要时聚焦：用于无训练视觉定位的自适应路由与协作定位

Focus When Necessary: Adaptive Routing and Collaborative Grounding for Training-Free Visual Grounding

East China University of Science and Technology（华东理工大学）；Tsinghua University（清华大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Science and Technology of China（中国科学技术大学）

Yifan Wang, Peiming Li, Shiyu Li, Zhiyuan Hu, Xiaochen Yang, Wenming Yang, Yang Tang, Zheng Wei

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出LazyMCoT动态框架，通过自适应路由评估不确定性，对简单查询跳过处理，对困难样本利用协作定位模块进行两阶段精炼，在提升推理精度的同时降低平均推理延迟。

摘要：

虽然多模态大语言模型（MLLMs）在跨模态推理方面表现出色，但它们通常难以感知复杂高分辨率图像中的细粒度细节。最近的无训练方法通过图像缩放和局部裁剪来解决这一问题。然而，不加区分地应用这些操作会导致简单查询的计算冗余，并且可能因截断必要的全局上下文或引入无关的背景噪声而降低准确性。为此，我们提出了LazyMCoT，一个动态且无需训练的框架，能够根据样本难度自适应地分配视觉定位工作。该框架具有自适应路由机制，通过单次前向传递的首词统计量来评估预测不确定性。这有效地绕过了置信度高的案例，同时通过保形校准确保困难样本的召回。对于这些具有挑战性的案例，协作定位模块通过两阶段精炼过程，将模型固有的跨模态注意力与外部视觉专家相结合。该精炼过程生成精确的局部显示，以恢复小目标或被遮挡的目标。在多个基准上的大量实验表明，LazyMCoT通过同时提高推理精度和降低平均推理延迟，与基于训练的方法相媲美。我们的代码可在https://github.com/TencentBAC/LazyMCoT获取。

英文摘要：

While Multimodal Large Language Models (MLLMs) excel in cross-modal reasoning, they often struggle to perceive fine-grained details in complex high-resolution images. Recent training-free methods address this through image scaling and localized cropping. However, applying these manipulations indiscriminately introduces computational redundancy for simple queries and can degrade accuracy by truncating essential global context or introducing irrelevant background noise. To this end, we propose LazyMCoT, a dynamic and training-free framework that adaptively allocates visual grounding efforts based on sample difficulty. The framework features an Adaptive Routing mechanism that evaluates predictive uncertainty using first-token statistics from a single forward pass. This efficiently bypasses confident cases while ensuring the recall of difficult samples via conformal calibration. For these challenging cases, a Collaborative Grounding module integrates the inherent cross-modal attention of the model with an external visual expert through a two-stage refinement process. This refinement process generates a precise localized display to recover small or occluded targets. Extensive experiments across diverse benchmarks demonstrate that LazyMCoT rivals training-based approaches by simultaneously improving reasoning accuracy and reducing average inference latency. Our code is availble at https://github.com/TencentBAC/LazyMCoT.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文