发表机构
Zhejiang University; Southeast University（浙江大学; 东南大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对大型视觉语言模型的幻觉问题，提出无需训练的推理时方法动态对齐补偿，结合分层语义补偿与序列语义校正，在9个多模态基准上可减少幻觉并保持整体性能。
AI 中文摘要
大型视觉语言模型（LVLMs）仍易出现幻觉，产生与多模态输入无关或不一致的响应。现有缓解方法主要依赖外部监督、输出校准或注意力调节，却未充分探索自回归生成的内部表示动态。我们发现一种推理时的失败模式：跨模态表示在解码器各层退化，且随生成步骤漂移，导致 token 预测不稳定，增加幻觉风险。我们提出动态对齐补偿（DAC），一种无需训练的推理时方法，可检测表示发散并选择性应用轻量残差补偿。DAC 结合分层语义补偿以缓解层间退化，以及序列语义校正以约束时间漂移。在多个 LVLM 骨干网络的 9 个幻觉聚焦及通用多模态基准上的实验表明，DAC 可持续减少幻觉，同时保持强大的整体性能。
英文摘要
Large Vision-Language Models (LVLMs) remain prone to hallucinations, producing responses that are irrelevant or inconsistent with the multimodal input. Existing mitigation methods mainly rely on external supervision, output calibration, or attention regulation, leaving the internal representation dynamics of autoregressive generation underexplored. We identify an inference-time failure mode in which cross-modal representations degrade across decoder layers and drift across generation steps, destabilizing token prediction and increasing hallucination risk. We propose \emph{Dynamic Alignment Compensation} (DAC), a training-free inference-time method that detects representation divergence and selectively applies lightweight residual compensation. DAC combines Layer-wise Semantic Compensation to mitigate inter-layer degradation with Sequential Semantic Correction to constrain temporal drift. Experiments on nine hallucination-focused and general-purpose multimodal benchmarks across multiple LVLM backbones show that DAC consistently reduces hallucinations while maintaining strong overall performance.
CommentsAccepted by EMNLP2026 Findings
