现在行动：通过自适应上下文整合预防治愈LVLM幻觉
ACT Now: Preempting LVLM Hallucinations via Adaptive Context Integration
AI总结：
本文提出ACT方法，通过自适应整合上下文信息缓解LVLM幻觉问题，采用视觉上下文探索和语义上下文聚合技术，有效减少幻觉并提升生成性能。
摘要：
大型视觉-语言模型（LVLMs）常面临严重的幻觉问题。现有缓解策略主要依赖孤立的单步状态来增强视觉聚焦或抑制强语言先验。然而，这些静态方法忽略了生成过程中动态上下文变化，难以纠正继承的信息损失。为解决这一限制，我们提出自适应上下文整合（ACT），一种无需训练的推理干预方法，通过自适应整合上下文信息来缓解幻觉。具体而言，我们首先提出视觉上下文探索，利用时空分析来自适应放大负责视觉探索的注意力头。为进一步促进视觉-语言对齐，我们提出语义上下文聚合，消除潜在语义查询以有效聚合视觉证据，从而解决由离散令牌预测导致的信息损失。在多样化的LVLMs上进行的广泛实验表明，ACT显著减少了幻觉，并在判别和生成基准上取得了具有竞争力的结果，作为稳健且高度适应的解决方案，不牺牲基本生成能力。
英文摘要：
Large Vision-Language Models (LVLMs) frequently suffer from severe hallucination issues. Existing mitigation strategies predominantly rely on isolated, single-step states to enhance visual focus or suppress strong linguistic priors. However, these static approaches neglect dynamic context changes across the generation process and struggles to correct inherited information loss. To address this limitation, we propose Adaptive Context inTegration (ACT), a training-free inference intervention method that mitigates hallucination through the adaptive integration of contextual information. Specifically, we first propose visual context exploration, which leverages spatio-temporal profiling to adaptively amplify attention heads responsible for visual exploration. To further facilitate vision-language alignment, we propose semantic context aggregation that marginalizes potential semantic queries to effectively aggregate visual evidence, thereby resolving the information loss caused by the discrete nature of token prediction. Extensive experiments across diverse LVLMs demonstrate that ACT significantly reduces hallucinations and achieves competitive results on both discriminative and generative benchmarks, acting as a robust and highly adaptable solution without compromising fundamental generation capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
