看到正确却说错：在MLLMs中无需训练的跨层和内层细化
Seeing Right but Saying Wrong: Inter- and Intra-Layer Refinement in MLLMs without Training
AI总结：
DualPD通过双视角解码细化策略，无需额外训练提升多模态大语言模型的视觉理解准确性。
摘要：
多模态大语言模型（MLLMs）在多种视觉-语言任务中展现出了强大的能力。然而，其内部推理常常表现出关键的不一致：尽管更深层的层可能关注正确的视觉区域，最终的预测却常常受到早期层中噪声注意力的误导。这导致了模型内部理解与最终表达之间的脱节，我们将其描述为“看到正确却说错”。为了解决这个问题，我们提出了DualPD，一种双视角解码细化策略，能够在不进行额外训练的情况下增强视觉理解。DualPD由两个组成部分组成。（1）层间注意力引导的对比日志模块通过比较表现出最大注意力偏移的层之间的输出日志，捕捉信念在正确答案上的演变。（2）头部信息过滤模块抑制那些关注无关区域的低贡献注意力头部，从而在每层内提高注意力质量。在多个多模态基准测试中，对LLaVA和Qwen-VL模型家族进行的实验表明，DualPD在不训练的情况下持续提高准确性，证实了其有效性和通用性。代码将在发表时发布。
英文摘要：
Multimodal Large Language Models (MLLMs) have demonstrated strong capabilities across a variety of vision-language tasks. However, their internal reasoning often exhibits a critical inconsistency: although deeper layers may attend to the correct visual regions, final predictions are frequently misled by noisy attention from earlier layers. This results in a disconnect between what the model internally understands and what it ultimately expresses, a phenomenon we describe as seeing it right but saying it wrong. To address this issue, we propose DualPD, a dual-perspective decoding refinement strategy that enhances the visual understanding without any additional training. DualPD consists of two components. (1) The layer-wise attention-guided contrastive logits module captures how the belief in the correct answer evolves by comparing output logits between layers that exhibit the largest attention shift. (2) The head-wise information filtering module suppresses low-contribution attention heads that focus on irrelevant regions, thereby improving attention quality within each layer. Experiments conducted on both the LLaVA and Qwen-VL model families across multiple multimodal benchmarks demonstrate that DualPD consistently improves accuracy without training, confirming its effectiveness and generalizability. The code will be released upon publication.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
