Despite the advanced capabilities of Large Vision-Language Models (LVLMs), they frequently suffer from object hallucination. One reason is that visual features and pretrained textual representations often become intertwined in the deeper network layers. To address this, we propose REVIS, a training-free framework designed to explicitly re-activate this suppressed visual information. Rooted in latent space geometry, REVIS extracts the pure visual information vector via orthogonal projection and employs a calibrated strategy to perform sparse intervention only at the precise depth where suppression occurs. This surgical approach effectively restores visual information with minimal computational cost. Empirical evaluations on standard benchmarks demonstrate that REVIS reduces object hallucination rates by approximately 19% compared to state-of-the-art baselines, while preserving general reasoning capabilities.
Mitigating Entangled Steering in Large Vision-Language Models for Hallucination Reduction
Ministry of Education Key Laboratory of Intelligent Networks and Network Security(教育部智能网络与网络安全重点实验室);Centre for Frontier AI Research, Institute of High Performance Computing, Agency for Science, Technology and Research(前沿人工智能研究中心,高性能计算研究所,科技研究局)
Beyond Visual Enhancement: Adaptive Multi-Context Steering to Mitigate LVLM Hallucinations
Shanghai Jiao Tong University(上海交通大学);Peking University(北京大学);Beijing University of Chemical Technology(北京化工大学);Nanjing University of Aeronautics and Astronautics(南京航空航天大学);University of Science and Technology of China(中国科学技术大学);Tsinghua University(清华大学)
Steering LVLMs via Sparse Autoencoder for Hallucination Mitigation
School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University)(东南大学新一代人工智能技术及其交叉应用关键实验室);Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所基础模型研究中心);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系);Wuhan University of Technology(武汉理工大学);National University of Singapore(新加坡国立大学)
SAVE: Sparse Autoencoder-Driven Visual Information Enhancement for Mitigating Object Hallucination
Department of Electrical and Computer Engineering, Seoul National University(电子与计算机工程系,首尔国立大学);Daegu Gyeongbuk Institute of Science and Technology(大邱庆州科学技术院);IPAI, AIIS, ASRI, INMC, and ISRC, Seoul National University(IPAI、AIIS、ASRI、INMC 和 ISRC,首尔国立大学)
Adaptive Residual-Update Steering for Low-Overhead Hallucination Mitigation in Large Vision Language Models
Aalto University, Espoo, Finland(艾尔沃大学,芬兰 Espoo);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China(深圳先进技术研究院,中国科学院,深圳)
Prefill-Time Intervention for Mitigating Hallucination in Large Vision-Language Models
University of Science and Technology of China(中国科学技术大学);Shanghai Advanced Research Institute, Chinese Academy of Sciences(上海先进研究院,中国科学院);University of Chinese Academy of Sciences(中国科学院大学)
Whisper Hallucination Detection and Mitigation via Hidden Representation Steering and Sparse AutoEncoders
AI Foundation and Algorithm Lab(AI基础与算法实验室);National University of Science and Technology MISIS(科学与技术国立大学MISIS);National Research University Higher School of Economics(国家研究大学经济高等学院)
When Personalization Misleads: Understanding and Mitigating Hallucinations in Personalized LLMs
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学首都人工智能学院);AI Lab at Lenovo Research(联想研究院人工智能实验室);School of Artificial Intelligence and Data Science, University of International Business and Economics(国际商务与经济大学人工智能与数据科学学院)
Toward universal steering and monitoring of AI models
Computer Science and Engineering(计算机科学与工程);Broad Institute of MIT and Harvard(MIT和哈佛大学Broad研究所);UC San Diego(圣地亚哥大学);Harvard SEAS(哈佛大学工程与应用科学学院);MIT Mathematics(MIT数学系);Halıcıoğlu Data Science Institute(Halıcıoğlu数据科学研究所);Harvard CMSA(哈佛大学计算机科学与应用数学系)
Before the Token Commits: Trajectory-Level Benchmarking of Visual Hallucinations in Diffusion VLMs
Nanjing University of Aeronautics and Astronautics(南京航空航天大学);State Key Laboratory of Ocean Sensing, ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University(浙江大学杭州国际科创中心海洋感知全国重点实验室);Security Capability Center(360安全能力中心)
Hallucination Detection-Guided Preference Optimization for Clinical Summarization
Manning College of Information and Computer Sciences(Manning信息与计算机科学学院);Ensemble HP;Columbia University(哥伦比亚大学);University of Massachusetts Amherst(马萨诸塞大学阿姆赫斯特分校)
Hallucination-Resistant Relation Extraction via Dependency-Aware Sentence Simplification and Two-tiered Hierarchical Refinement
Shanghai Jiao Tong University(上海交通大学);University of California San Diego(加州大学圣地亚哥分校);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Alibaba Group(阿里巴巴集团)
Internal Activation Revision: Safeguarding Vision Language Models Without Parameter Update
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Fraunhofer FOKUS(弗劳恩霍夫协会FOKUS研究所);The University of Tokyo(东京大学);University of Alberta(阿尔伯塔大学)
Revisiting KRISP: A Lightweight Reproduction and Analysis of Knowledge-Enhanced Vision-Language Models
Centre For Digital Health(数字健康中心);Indian Institute of Technology Bombay(孟买印度理工学院);Department of Metallurgical engineering & Material science(冶金工程与材料科学系);Department of Bioscience & Bioengineering(生物科学与生物工程系)
Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Shanghai Jiao Tong University(上海交通大学);The University of Sydney(悉尼大学);Renmin University of China(中国人民大学);University of Chinese Academy of Sciences(中国科学院大学)