VCE：通过视觉对比编辑实现LVLMs的零成本幻觉抑制方法
VCE: A zero-cost hallucination mitigation method of LVLMs via visual contrastive editing
AI总结：
本文提出VCE方法，通过分析模型对对比视觉扰动的响应，利用SVD分解激活模式以抑制幻觉倾向，有效减少多基准测试中的物体幻觉，同时保持计算效率。
摘要：
大型视觉-语言模型（LVLMs）经常出现物体幻觉（OH）问题，即生成包含输入图像中不存在的物体的描述。这种现象在医疗影像和自动驾驶等实际应用中尤为严重，因为准确性至关重要。最近的研究表明，幻觉问题可能源于语言先验：在预训练过程中学习的偏见，导致LVLMs根据统计共现性生成词语。为缓解此问题，我们提出了视觉对比编辑（VCE），一种新颖的后处理方法，通过分析模型对对比视觉扰动的响应，识别并抑制幻觉倾向。利用奇异值分解（SVD），我们分解模型的激活模式以隔离幻觉子空间，并应用针对性的参数编辑以减弱其影响。与需要微调或标记数据的现有方法不同，VCE作为一种无标签干预，使其在资源受限环境中具有可扩展性和实用性。实验结果表明，VCE在多个基准测试中有效减少了物体幻觉，同时保持模型的原始计算效率。
英文摘要：
Large vision-language models (LVLMs) frequently suffer from Object Hallucination (OH), wherein they generate descriptions containing objects that are not actually present in the input image. This phenomenon is particularly problematic in real-world applications such as medical imaging and autonomous driving, where accuracy is critical. Recent studies suggest that the hallucination problem may stem from language priors: biases learned during pretraining that cause LVLMs to generate words based on their statistical co-occurrence. To mitigate this problem, we propose Visual Contrastive Editing (VCE), a novel post-hoc method that identifies and suppresses hallucinatory tendencies by analyzing the model's response to contrastive visual perturbations. Using Singular Value Decomposition (SVD), we decompose the model's activation patterns to isolate hallucination subspaces and apply targeted parameter edits to attenuate its influence. Unlike existing approaches that require fine-tuning or labeled data, VCE operates as a label-free intervention, making it both scalable and practical for deployment in resource-constrained settings. Experimental results demonstrate that VCE effectively reduces object hallucination across multiple benchmarks while maintaining the model's original computational efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
