HalluciDoctor：缓解视觉指令数据中的幻觉毒性
HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data
AI总结：
本文提出基于交叉检查范式的幻觉检测与消除框架HalluciDoctor，通过自动消除训练数据中的幻觉并利用反事实指令扩展平衡数据分布，成功缓解了44.6%的幻觉，增强了多模态大语言模型对幻觉的抵抗力。
摘要：
在机器生成的指令跟随数据上微调的多模态大语言模型（MLLMs）在各种多模态理解和生成任务中展现出了卓越的性能。然而，机器生成数据中固有的幻觉可能导致MLLMs产生幻觉输出，这一现象仍有待深入探索。本工作旨在调查各种幻觉（即对象、关系、属性幻觉），并缓解大规模机器生成视觉指令数据集中的幻觉毒性。借鉴人类识别事实错误的能力，我们提出了一种基于交叉检查范式的新型幻觉检测与消除框架HalluciDoctor。我们使用该框架自动识别并消除训练数据中的幻觉。有趣的是，HalluciDoctor还指出，长尾对象共现引起的虚假相关性是导致幻觉的原因之一。基于此，我们执行反事实视觉指令扩展以平衡数据分布，从而增强MLLMs对幻觉的抵抗力。在幻觉评估基准上的综合实验表明，与LLaVA相比，我们的方法成功相对缓解了44.6%的幻觉，并保持了具有竞争力的性能。本文的数据和代码已公开可用：https://github.com/Yuqifan1117/HalluciDoctor。
英文摘要：
Multi-modal Large Language Models (MLLMs) tuned on machine-generated instruction-following data have demonstrated remarkable performance in various multi-modal understanding and generation tasks. However, the hallucinations inherent in machine-generated data, which could lead to hallucinatory outputs in MLLMs, remain under-explored. This work aims to investigate various hallucinations (i.e., object, relation, attribute hallucinations) and mitigate those hallucinatory toxicities in large-scale machine-generated visual instruction datasets. Drawing on the human ability to identify factual errors, we present a novel hallucination detection and elimination framework, HalluciDoctor, based on the cross-checking paradigm. We use our framework to identify and eliminate hallucinations in the training data automatically. Interestingly, HalluciDoctor also indicates that spurious correlations arising from long-tail object co-occurrences contribute to hallucinations. Based on that, we execute counterfactual visual instruction expansion to balance data distribution, thereby enhancing MLLMs' resistance to hallucinations. Comprehensive experiments on hallucination evaluation benchmarks show that our method successfully mitigates 44.6% hallucinations relatively and maintains competitive performance compared to LLaVA. The data and code for this paper are publicly available. \url{https://github.com/Yuqifan1117/HalluciDoctor}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
