CHASD：面向LVLMs中幻觉的语言增量校准对比解码
CHASD: Language Increment-Calibrated Contrastive Decoding against Hallucination in LVLMs
AI总结：
提出CHASD框架，通过不确定性驱动的置信门控和注意力引导的局部扰动，在推理时按需校准对比解码，减少对象幻觉并提升效率。
摘要：
大型视觉-语言模型展现了强大的多模态推理能力，但当语言先验主导不足或错位的视觉证据时，它们仍然容易产生对象幻觉。无训练对比解码方法通过比较原始和扰动视觉输入的预测来缓解此问题，但现有方法要么应用可能改变有用视觉证据的全局扰动，要么在每个解码步骤调用额外的负分支。在本文中，我们观察到幻觉风险是瞬态且特定于token的：视觉注意力在生成的token间转移，而一些功能token以高置信度产生，不需要对比校准。基于这一观察，我们提出面向大型视觉-语言模型的对比幻觉感知逐步解码（CHASD），一种“按需校准”的推理时框架。CHASD使用不确定性驱动的置信门控，仅当下一token的最大概率低于阈值时激活对比分支，并通过注意力引导的局部扰动构建负分支，扰动当前显著的视觉token。这种设计减少了不必要的负分支前向传播，同时保留了高置信度步骤的原始分布。在POPE、AMBER、MME、MMHal-Bench和CHAIR上的实验表明，CHASD在强无训练基线上改进了幻觉相关指标，并具有有竞争力的推理效率。
英文摘要：
Large Vision-Language Models have shown strong multimodal reasoning capabilities, yet they remain susceptible to object hallucinations when language priors dominate insufficient or misaligned visual evidence. Training-free contrastive decoding methods mitigate this issue by comparing predictions from original and perturbed visual inputs, but existing approaches either apply global perturbations that may alter useful visual evidence or invoke an additional negative branch at every decoding step. In this paper, we observe that hallucination risks are transient and token-specific: visual attention shifts across generated tokens, while some functional tokens are produced with high confidence and do not require contrastive calibration. Based on this observation, we propose Contrastive Hallucination-Aware Step-wise Decoding (CHASD) for Large Vision-Language Models, an inference-time framework for "calibration on demand". CHASD uses an uncertainty-driven confidence gate to activate the contrastive branch only when the maximum probability of the next-token is less than the threshold, and constructs the negative branch through attention-guided localized perturbations of the currently salient visual tokens. This design reduces unnecessary negative-branch forward passes while preserving the original distribution for high-confidence steps. Experiments on POPE, AMBER, MME, MMHal-Bench, and CHAIR show that CHASD improves hallucination-related metrics over strong training-free baselines with competitive inference efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
