HalluRNN: 通过大规模视觉-语言模型中的递归跨层推理缓解幻觉
HalluRNN: Mitigating Hallucinations via Recurrent Cross-Layer Reasoning in Large Vision-Language Models
AI总结：
HalluRNN通过递归跨层推理模块缓解大规模视觉-语言模型中的幻觉问题，提升模型稳定性和性能。
摘要：
尽管大规模视觉-语言模型（LVLMs）在各种任务上取得了显著性能，但它们仍然容易产生幻觉——生成文本合理但视觉上不成立的输出。尽管先前的方法通常通过数据驱动的微调或创新的解码策略来解决这个问题，但这些方法往往需要大量资源或任务特定的配置。在本工作中，我们提出了一种架构层面的解决方案，即HalluRNN，通过递归跨层推理增强模型稳定性。具体而言，我们提出了一种新的双门深度传播单元（DG-DPU）模块，该模块在各层之间共享并递归地细化隐藏状态。这使得信息能够在模型中自适应传播，强制各层之间的一致性，并缓解由表征漂移引起的幻觉。通过仅微调DG-DPU模块，HalluRNN在多个基准上实现了强大而稳健的性能。
英文摘要：
Though Large Vision-Language Models (LVLMs) have achieved remarkable performance across various tasks, they are still prone to hallucinations-generating outputs that are textually plausible but visually ungrounded. While prior approaches generally address this issue through data-centric fine-tuning or innovative decoding strategies, these methods often require substantial resources or task-specific configurations. In this work, we introduce an architecture-level solution, HalluRNN, which enhances model stability through recurrent cross-layer reasoning. Specifically, we propose a novel Dual-Gated Depth Propagation Unit (DG-DPU) module, which is shared across layers and recurrently refines hidden states. This allows for the adaptive propagation of information throughout the model, enforces consistency across layers, and mitigates hallucinations caused by representational drift. By fine-tuning only the DG-DPU module, HalluRNN achieves strong and robust performance across multiple benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
