Recent advancements in Large Vision-Language Models (LVLMs) have significantly expanded their utility in tasks like image captioning and visual question answering. However, they still struggle with object hallucination, where models generate descriptions that inaccurately reflect the visual content by including nonexistent objects or misrepresenting existing ones. While previous methods, such as data augmentation and training-free approaches, strive to tackle this issue, they still encounter scalability challenges and often depend on additional external modules. In this work, we propose Ensemble Decoding (ED), a novel strategy that splits the input image into sub-images and combines logit distributions by assigning weights through the attention map. Furthermore, we introduce ED adaptive plausibility constraint to calibrate logit distribution and FastED, a variant designed for speed-critical applications. Extensive experiments across hallucination benchmarks demonstrate that our proposed method achieves state-of-the-art performance, validating the effectiveness of our approach.
MHSA: A Lightweight Framework for Mitigating Hallucinations via Steered Attention in LVLMs
Tsinghua University（清华大学）；Tsinghua University, Tencent（清华大学腾讯）；Tencent（腾讯）；University of Science and Technology Beijing（北京科技大学）；University of Macau（澳门大学）
Prescribing the Right Remedy: Mitigating Hallucinations in Large Vision-Language Models via Targeted Instruction Tuning
Beijing Key Lab of Traffic Data Analysis and Mining（北京交通大数据分析与挖掘重点实验室）；Beijing Jiaotong University（北京交通大学）；School of Information Technology and Management（信息科学技术学院）
SeeMe: Mitigating Hallucinations in Large Vision-Language Models through Effective Visual Token Engineering
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（北京大学计算机科学学院多媒体信息处理国家重点实验室）；University of Pennsylvania（宾夕法尼亚大学）；University of Electronic Science and Technology of China（电子科技大学）；Nanyang Technological University（南洋理工大学）；Tsinghua University（清华大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；De Artificial Intelligence Lab（德人工智能实验室）
Mitigating Entangled Steering in Large Vision-Language Models for Hallucination Reduction
Ministry of Education Key Laboratory of Intelligent Networks and Network Security（教育部智能网络与网络安全重点实验室）；Centre for Frontier AI Research, Institute of High Performance Computing, Agency for Science, Technology and Research（前沿人工智能研究中心，高性能计算研究所，科技研究局）
SAVE: Sparse Autoencoder-Driven Visual Information Enhancement for Mitigating Object Hallucination
Department of Electrical and Computer Engineering, Seoul National University（电子与计算机工程系，首尔国立大学）；Daegu Gyeongbuk Institute of Science and Technology（大邱庆州科学技术院）；IPAI, AIIS, ASRI, INMC, and ISRC, Seoul National University（IPAI、AIIS、ASRI、INMC 和 ISRC，首尔国立大学）
Prefill-Time Intervention for Mitigating Hallucination in Large Vision-Language Models
University of Science and Technology of China（中国科学技术大学）；Shanghai Advanced Research Institute, Chinese Academy of Sciences（上海先进研究院，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）
VIB-Probe: Detecting and Mitigating Hallucinations in Vision-Language Models via Variational Information Bottleneck
College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China（复旦大学计算机科学与人工智能学院，上海，中国）；Shanghai Key Laboratory of Intelligent Information Processing（上海智能信息处理重点实验室）
