HKVLM：通过将语言查询绑定到冻结检测器实现忠实推理定位
HKVLM: Faithful Query--Region Binding for Frozen-Detector Visual Grounding
AI总结：
提出HKVLM框架，通过将定位从语言路径中分离，利用冻结检测器和语言模型，仅训练轻量对齐钩子实现推理定位，在少样本冷启动场景下显著提升定位精度并减少幻觉。
摘要：
许多视觉请求——如“打开这个瓶子的物体”、“没有戴头盔的人”——需要推理，而不仅仅是类别匹配。纯开放词汇检测器需要明确的短语；视觉语言模型（VLM）能够推理，但“看得见却说错”，关注了正确区域却返回错误的框或标签。我们认为这是绑定失败：在坐标即文本的VLM中，定位通过自回归头传递，与语言生成耦合；在两阶段流程中，模型的意图被压缩到单个类别字符串中。我们提出HKVLM，将定位从语言路径中移除。一个冻结的、语言对齐的检测器生成与类别无关的区域提议；一个冻结的语言模型将推理指令编码为参考查询嵌入；一个轻量级的对齐钩子通过对比检索和二分匹配在共享嵌入空间中将查询绑定到区域。一个基于感知的忠实性否决机制禁止命名没有区域支持的物体。仅训练钩子，针对单体VLM调优困难的小数据冷启动场景。我们形式化了“说与看”分解，将定位错误（SeeErr）与绑定错误（SayErr）分离，并在RefCOCO/RefCOCO+/RefCOCOg和POPE上评估。使用冻结的Grounding DINO和Qwen2.5-VL，仅训练钩子将定位精度比未训练的跨空间匹配提升50-90倍；忠实性否决将POPE准确率从接近随机（0.50）提升到0.66-0.76，并将幻觉从约0.99降低到0.23-0.43，且仅需200个表达式。将提议数从M=50增加到M=300可在不重新训练的情况下将定位提升19-24%，证实剩余错误是感知性的（SeeErr）而非绑定性的（SayErr）。
英文摘要：
Visual grounding often fails even when the target object is present in the proposal pool, because the language-side referent is bound to the wrong region. We study this binding failure under frozen perception and ask whether an explicit query--region alignment hook, together with a perception-grounded abstention mechanism, can improve faithful grounding without retraining the detector or the vision-language backbone. HKVLM freezes a language-aligned open-vocabulary detector for localization and learns a lightweight hook that maps referential query embeddings to detector proposals in a shared space; a verifier abstains when no region sufficiently supports the query. We prove an exact proposal-level diagnostic decomposition, $(1-\mathrm{SeeErr})(1-\mathrm{SayErr})$, separating proposal-coverage failures from conditional binding failures, and a monotonicity result that characterizes the faithfulness--recall trade-off induced by abstention. Across RefCOCO, RefCOCO+, RefCOCOg, and POPE, HKVLM improves over untrained and trained matched-perception binding controls and substantially reduces hallucination through abstention. Strong coordinate-decoding and end-to-end fine-tuned baselines remain much higher in raw grounding accuracy, and a reasoning-stress set exposes binding as the main current bottleneck. We therefore present HKVLM as a diagnostic and mechanism-level study of query--region binding under frozen perception, not as an absolute localization leader.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
