Causal-LLaVA：用于缓解多模态大语言模型幻觉的因果解缠
Causal-LLaVA: Causal Disentanglement for Mitigating Hallucination in Multimodal Large Language Models
AI总结：
针对多模态大语言模型中的物体幻觉问题，提出因果解缠框架，通过因果驱动投影器和因果干预模块减少虚假相关性，在多个基准上显著降低幻觉并保持性能。
摘要：
多模态大语言模型（MLLMs）在视觉理解任务中表现出强大的性能，然而它们常常遭受物体幻觉的困扰——即生成与输入不一致或完全缺失的物体描述。这一问题与数据集偏差密切相关，其中物体频繁共现导致跨模态的语义表征纠缠。因此，模型可能错误地激活与输入常见关联但实际并不存在的物体表征。为解决这一问题，我们提出了一种因果驱动的解缠框架，通过因果干预来缓解幻觉。我们的方法包括视觉通路中的因果驱动投影器，以及集成到语言模型最终Transformer层中的因果干预模块。这些组件协同工作，以减少由有偏训练数据引起的虚假相关性。实验结果表明，我们的方法在多个多模态基准上显著减少了幻觉，同时保持了强劲的性能。可视化分析进一步证实了物体表征的可分离性得到改善。代码可在以下网址获取：https://github.com/IgniSavium/Causal-LLaVA
英文摘要：
Multimodal Large Language Models (MLLMs) have demonstrated strong performance in visual understanding tasks, yet they often suffer from object hallucinations--generating descriptions of objects that are inconsistent with or entirely absent from the input. This issue is closely related to dataset biases, where frequent co-occurrences of objects lead to entangled semantic representations across modalities. As a result, models may erroneously activate object representations that are commonly associated with the input but not actually present. To address this, we propose a causality-driven disentanglement framework that mitigates hallucinations through causal intervention. Our approach includes a Causal-Driven Projector in the visual pathway and a Causal Intervention Module integrated into the final transformer layer of the language model. These components work together to reduce spurious correlations caused by biased training data. Experimental results show that our method significantly reduces hallucinations while maintaining strong performance on multiple multimodal benchmarks. Visualization analyses further confirm improved separability of object representations. The code is available at: https://github.com/IgniSavium/Causal-LLaVA
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
