Mitigating Entangled Steering in Large Vision-Language Models for Hallucination Reduction
Ministry of Education Key Laboratory of Intelligent Networks and Network Security（教育部智能网络与网络安全重点实验室）；Centre for Frontier AI Research, Institute of High Performance Computing, Agency for Science, Technology and Research（前沿人工智能研究中心，高性能计算研究所，科技研究局）
Large Vision-Language Models (LVLMs) have achieved remarkable success across cross-modal tasks but remain hindered by hallucinations, producing textual outputs inconsistent with visual content. Existing methods mitigate hallucinations but often alter generation behavior, resulting in shorter outputs and shifted token distributions, especially in latent space steering approaches. We identify that this issue stems from entangled steering signals, where suppressing hallucinations inadvertently disrupts the model's intrinsic generation behavior. To address this, we propose MESA, an effective plug-and-play framework that performs controlled and selective latent intervention for hallucination mitigation. Specifically, MESA targets hallucination-relevant responses while preserving the model's original token distribution, enabling effective hallucination reduction without compromising generation behavior. Extensive experiments across diverse generative and discriminative benchmarks demonstrate that MESA consistently reduces hallucinations while better preserving generation behavior, outperforming prior methods across multiple LVLM families.
Beyond Visual Enhancement: Adaptive Multi-Context Steering to Mitigate LVLM Hallucinations
Shanghai Jiao Tong University（上海交通大学）；Peking University（北京大学）；Beijing University of Chemical Technology（北京化工大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；University of Science and Technology of China（中国科学技术大学）；Tsinghua University（清华大学）
MHSA: A Lightweight Framework for Mitigating Hallucinations via Steered Attention in LVLMs
Tsinghua University（清华大学）；Tsinghua University, Tencent（清华大学腾讯）；Tencent（腾讯）；University of Science and Technology Beijing（北京科技大学）；University of Macau（澳门大学）
Prescribing the Right Remedy: Mitigating Hallucinations in Large Vision-Language Models via Targeted Instruction Tuning
Beijing Key Lab of Traffic Data Analysis and Mining（北京交通大数据分析与挖掘重点实验室）；Beijing Jiaotong University（北京交通大学）；School of Information Technology and Management（信息科学技术学院）
SeeMe: Mitigating Hallucinations in Large Vision-Language Models through Effective Visual Token Engineering
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（北京大学计算机科学学院多媒体信息处理国家重点实验室）；University of Pennsylvania（宾夕法尼亚大学）；University of Electronic Science and Technology of China（电子科技大学）；Nanyang Technological University（南洋理工大学）；Tsinghua University（清华大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；De Artificial Intelligence Lab（德人工智能实验室）
SAVE: Sparse Autoencoder-Driven Visual Information Enhancement for Mitigating Object Hallucination
Department of Electrical and Computer Engineering, Seoul National University（电子与计算机工程系，首尔国立大学）；Daegu Gyeongbuk Institute of Science and Technology（大邱庆州科学技术院）；IPAI, AIIS, ASRI, INMC, and ISRC, Seoul National University（IPAI、AIIS、ASRI、INMC 和 ISRC，首尔国立大学）
Prefill-Time Intervention for Mitigating Hallucination in Large Vision-Language Models
University of Science and Technology of China（中国科学技术大学）；Shanghai Advanced Research Institute, Chinese Academy of Sciences（上海先进研究院，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）
VIB-Probe: Detecting and Mitigating Hallucinations in Vision-Language Models via Variational Information Bottleneck
College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China（复旦大学计算机科学与人工智能学院，上海，中国）；Shanghai Key Laboratory of Intelligent Information Processing（上海智能信息处理重点实验室）
