FINER：MLLMs在细粒度负查询下产生幻觉
FINER: MLLMs Hallucinate under Fine-grained Negative Queries
AI总结：
本文提出FINER基准测试，分析MLLMs在细粒度不匹配与真实元素共存时的幻觉问题，并通过FINER-Tuning提升模型性能。
摘要：
多模态大语言模型（MLLMs）在处理细粒度查询时面临幻觉挑战，现有基准测试对此关注不足。本文引入FINER，包含FINER-CompreCap和FINER-DOCCI两个基准。通过分析四个设置，发现幻觉源于细粒度不匹配与真实元素共存。提出FINER-Tuning，基于DPO优化，对四个前沿MLLMs进行微调，使幻觉减少达24.2%（InternVL3.5-14B），同时提升八个现有幻觉测试集性能，并增强六个基准的多模态能力。代码、基准和模型可在https://explainableml.github.io/finer-project/获取。
英文摘要：
Multimodal large language models (MLLMs) struggle with hallucinations, particularly with fine-grained queries, a challenge underrepresented by existing benchmarks that focus on coarse image-related questions. We introduce FIne-grained NEgative queRies (FINER), alongside two benchmarks: FINER-CompreCap and FINER-DOCCI. Using FINER, we analyze hallucinations across four settings: multi-object, multi-attribute, multi-relation, and ``what'' questions. Our benchmarks reveal that MLLMs hallucinate when fine-grained mismatches co-occur with genuinely present elements in the image. To address this, we propose FINER-Tuning, leveraging Direct Preference Optimization (DPO) on FINER-inspired data. Finetuning four frontier MLLMs with FINER-Tuning yields up to 24.2\% gains (InternVL3.5-14B) on hallucinations from our benchmarks, while simultaneously improving performance on eight existing hallucination suites, and enhancing general multimodal capabilities across six benchmarks. Code, benchmark, and models are available at \href{https://explainableml.github.io/finer-project/}{https://explainableml.github.io/finer-project/}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
