MIHBench：多模态大模型中多图像幻觉的基准测试与缓解
MIHBench: Benchmarking and Mitigating Multi-Image Hallucinations in Multimodal Large Language Models
AI总结：
该研究首次系统探究多图像MLLM的幻觉问题，提出含三项核心任务的MIHBench基准，明确幻觉发生的关键影响因素，并提出动态注意力平衡机制，有效减少多图像场景下的幻觉、提升语义整合与推理稳定性。
摘要：
尽管人们对多模态大模型（MLLM）中的幻觉问题日益关注，但现有研究主要聚焦于单图像场景，多图像场景下的幻觉问题在很大程度上仍未被探索。为填补这一空白，我们首次对多图像MLLM中的幻觉开展系统性研究，并提出MIHBench——一个专门用于评估跨多图像的物体相关幻觉的基准测试集。MIHBench包含三项核心任务：多图像物体存在性幻觉、多图像物体计数幻觉以及物体身份一致性幻觉，分别针对物体存在性语义理解、数量推理和跨视角身份一致性三个维度。通过大量评估，我们明确了与多图像幻觉发生相关的关键因素，包括：图像输入数量与幻觉发生概率之间存在递进关系；单图像幻觉倾向与多图像场景下观察到的幻觉倾向高度相关；同物体图像占比以及负样本在图像序列中的位置会影响物体身份一致性幻觉的发生。为应对这些挑战，我们提出一种动态注意力平衡机制，该机制可在保持整体视觉注意力比例的同时调整图像间注意力分布。在多个先进MLLM上开展的实验表明，我们的方法能有效减少幻觉发生，提升多图像场景下的语义整合与推理稳定性。
英文摘要：
Despite growing interest in hallucination in Multimodal Large Language Models, existing studies primarily focus on single-image settings, leaving hallucination in multi-image scenarios largely unexplored. To address this gap, we conduct the first systematic study of hallucinations in multi-image MLLMs and propose MIHBench, a benchmark specifically tailored for evaluating object-related hallucinations across multiple images. MIHBench comprises three core tasks: Multi-Image Object Existence Hallucination, Multi-Image Object Count Hallucination, and Object Identity Consistency Hallucination, targeting semantic understanding across object existence, quantity reasoning, and cross-view identity consistency. Through extensive evaluation, we identify key factors associated with the occurrence of multi-image hallucinations, including: a progressive relationship between the number of image inputs and the likelihood of hallucination occurrences; a strong correlation between single-image hallucination tendencies and those observed in multi-image contexts; and the influence of same-object image ratios and the positional placement of negative samples within image sequences on the occurrence of object identity consistency hallucination. To address these challenges, we propose a Dynamic Attention Balancing mechanism that adjusts inter-image attention distributions while preserving the overall visual attention proportion. Experiments across multiple state-of-the-art MLLMs demonstrate that our method effectively reduces hallucination occurrences and enhances semantic integration and reasoning stability in multi-image scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
