arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.11824cs.AIcs.LG
收藏

Revis: 通过稀疏潜在引导缓解大视觉-语言模型中的物体幻觉

Revis: Sparse Latent Steering to Mitigate Object Hallucination in Large Vision-Language Models

Ant Group(蚂蚁集团)

Jialin Wu, Wei Shi, Han Shen, Peigui Qi, Kunsheng Tang, Zhicong Huang, Binghao Wang, Zhou Yang

AI总结:

Revis通过潜在空间几何学提取纯视觉信息向量,采用校准策略在抑制发生深度进行稀疏干预,有效降低物体幻觉率19%,同时保持推理能力。

摘要:

尽管大型视觉-语言模型(LVLMs)具备先进能力,但经常出现物体幻觉问题。原因之一是视觉特征和预训练文本表示在深层网络中相互交织。为此,我们提出了REVIS,一种无需训练的框架,旨在显式重新激活被抑制的视觉信息。基于潜在空间几何学,REVIS通过正交投影提取纯视觉信息向量,并采用校准策略在抑制发生深度进行稀疏干预。这种手术方法以最小计算成本有效恢复视觉信息。在标准基准上的实验证明,与最先进的基线相比,REVIS将物体幻觉率降低了约19%,同时保持一般推理能力。

英文摘要:

Despite the advanced capabilities of Large Vision-Language Models (LVLMs), they frequently suffer from object hallucination. One reason is that visual features and pretrained textual representations often become intertwined in the deeper network layers. To address this, we propose REVIS, a training-free framework designed to explicitly re-activate this suppressed visual information. Rooted in latent space geometry, REVIS extracts the pure visual information vector via orthogonal projection and employs a calibrated strategy to perform sparse intervention only at the precise depth where suppression occurs. This surgical approach effectively restores visual information with minimal computational cost. Empirical evaluations on standard benchmarks demonstrate that REVIS reduces object hallucination rates by approximately 19% compared to state-of-the-art baselines, while preserving general reasoning capabilities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑