首个Logit提升：一种缓解大视觉-语言模型中物体幻觉的视觉 grounding 方法
First Logit Boosting: Visual Grounding Method to Mitigate Object Hallucination in Large Vision-Language Models
AI总结：
本文提出FLB方法，通过存储首个生成token的logit并加到后续预测中，缓解大视觉-语言模型中的长期衰减问题，有效减少物体幻觉并保持视觉信息。
摘要：
近年来，大型视觉-语言模型（LVLMs）在多种多模态任务中表现出色，但物体幻觉（生成不存在的物体）仍是一个持续挑战。尽管已有重新训练和外部grounding方法，但存在高数据成本或结构复杂性。训练自由方法如对比解码（CD）更高效，但仍有长期衰减问题，即视觉grounding减弱，语言先验主导。本文提出首个Logit提升（FLB），一种简单的训练自由技术，旨在缓解LVLMs中的长期衰减。FLB存储首个生成token的logit，并将其加到后续token预测中，有效缓解视觉信息的长期衰减。我们观察到，FLB（1）在整个生成过程中维持首个token中的视觉信息，（2）通过“the”token的稳定效应抑制幻觉词。实验结果表明，FLB在各种任务、基准和backbone模型中显著减少了物体幻觉。值得注意的是，它导致的推理开销极小，使其在实时多模态系统中具有高度应用价值。代码可在https://github.com/jiwooha20/FLB获取。
英文摘要：
Recent Large Vision-Language Models (LVLMs) have demonstrated remarkable performance across various multimodal tasks that require understanding both visual and linguistic inputs. However, object hallucination -- the generation of nonexistent objects in answers -- remains a persistent challenge. Although several approaches such as retraining and external grounding methods have been proposed to mitigate this issue, they still suffer from high data costs or structural complexity. Training-free methods such as Contrastive Decoding (CD) are more cost-effective, avoiding additional training or external models, but still suffer from long-term decay, where visual grounding weakens and language priors dominate as the generation progresses. In this paper, we propose First Logit Boosting (FLB), a simple yet effective training-free technique designed to alleviate long-term decay in LVLMs. FLB stores the logit of the first generated token and adds it to subsequent token predictions, effectively mitigating long-term decay of visual information. We observe that FLB (1) sustains the visual information embedded in the first token throughout generation, and (2) suppresses hallucinated words through the stabilizing effect of the ``The'' token. Experimental results show that FLB significantly reduces object hallucination across various tasks, benchmarks, and backbone models. Notably, it causes negligible inference overhead, making it highly applicable to real-time multimodal systems. Code is available at https://github.com/jiwooha20/FLB
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
