提出并关注：通过多令牌局部注意力实现无训练的多模态大语言模型接地置信度
Propose and Attend: Training-free MLLM Grounding Confidence via Multi-Token Localized Attention
AI总结：
研究多模态大语言模型接地置信度问题，提出无训练的多令牌局部注意力（MTLA）方法，通过在声称区域内求和并跨预测令牌聚合恢复更强接地信号，在多模型家族和模态中提升了幻觉AUROC及零样本检测AP。
摘要：
多模态大语言模型能够发出局部预测，如物体的边界框以及视频和音频事件的时间窗口，但它们经常产生幻觉。模型自身的令牌对数概率几乎没有信息量，将接地质量与输入模糊性混为一谈，一旦模型做出决策，坐标令牌就变得几乎确定。我们提出了多令牌局部注意力（MTLA）：一种无训练的事后分数，用于衡量预测令牌对它们所声称区域的关注程度。基于注意力的先前检测器在整个输入模态上求和注意力并读取单个响应令牌，是较弱的特殊情况；我们表明仅在声称区域内求和并跨所有预测令牌聚合可恢复更强的接地信号。同样的方法几乎可以轻松应用于其他模态和任务。在多个多模态大语言模型家族和三种模态中，MTLA比最佳先前无训练基线将幻觉AUROC提高了7到38。用作重新排序的置信度分数时，它几乎使开源8B通用模型的零样本COCO检测AP翻倍，缩小了与无任何特定任务训练的监督检测器的差距。
英文摘要：
Multimodal large language models can emit localized predictions, bounding boxes for objects and temporal windows for video and audio events, but they hallucinate these regions prolifically. The model's own token log-probabilities are nearly uninformative: they conflate grounding quality with input ambiguity, and coordinate tokens become near-deterministic once the model commits. We propose Multi-Token Localized Attention (MTLA): a training-free, post-hoc score that measures how strongly a prediction's tokens attend to the region they claim. Prior attention-based detectors, which sum attention over the entire input modality and read a single response token, are weaker special cases; we show that summing only within the claimed region and aggregating across all prediction tokens recovers a stronger grounding signal. The same recipe applies almost trivially to other modalities and tasks: object detection in images and temporal localization in video and audio. Across multiple MLLM families and three modalities, MTLA improves hallucination AUROC by +7 to +38 over the best prior training-free baseline. Used as a confidence score for re-ranking, it nearly doubles the zero-shot COCO detection AP of an open-source 8B generalist (from 20.4 to 37.0), narrowing the gap to supervised detectors without any task-specific training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
