SECOND: Mitigating Perceptual Hallucination in Vision-Language Models via Selective and Contrastive Decoding
Department of Electrical and Computer Engineering, Seoul National University（首尔大学电气与计算机工程系）；Interdisciplinary Program in Artificial Intelligence, Seoul National University（首尔大学人工智能跨学科项目）
Despite significant advancements in Vision-Language Models (VLMs), the performance of existing VLMs remains hindered by object hallucination, a critical challenge to achieving accurate visual understanding. To address this issue, we propose SECOND: Selective and Contrastive Decoding, a novel approach that enables VLMs to effectively leverage multi-scale visual information with an object-centric manner, closely aligning with human visual perception. SECOND progressively selects and integrates multi-scale visual information, facilitating a more precise interpretation of images. By contrasting these visual information iteratively, SECOND significantly reduces perceptual hallucinations and outperforms a wide range of benchmarks. Our theoretical analysis and experiments highlight the largely unexplored potential of multi-scale application in VLMs, showing that prioritizing and contrasting across scales outperforms existing methods.
Seeing Clearly without Training: Mitigating Hallucinations in Multimodal LLMs for Remote Sensing
School of Computer Science, Wuhan University, China（武汉大学计算机学院）；Zhongguancun Academy, China（中关村学院）；School of Computer Science, Chongqing University, China（重庆大学计算机学院）
Understanding and Mitigating Hallucinations in Multimodal Chain-of-Thought Models
School of Computer Science and Ningbo Institute, Northwestern Polytechnical University, China（西北工业大学计算机学院和宁波研究院）；National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, China（国家空天地海一体化大数据应用技术国家工程实验室）
HIME: Mitigating Object Hallucinations in LVLMs via Hallucination Insensitivity Model Editing
School of Information and Communication Technology, Griffith University, Australia（信息与通信技术学院，格里菲斯大学）；Data61, CSIRO, Australia（Data61，澳大利亚联邦科学与工业研究组织）；School of Engineering, Macquarie University, Sydney, Australia（工程学院，麦觉大学）；School of Information Systems, Queensland University of Technology, Australia（信息系统学院，昆士兰技术大学）
Beyond Attention Imbalance: Mitigating Hallucinations via Spectral Surgery
National University of Defense Technology（国防科技大学）；Northwestern Polytechnical University（西北工业大学）；National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean（空天海一体化国家工程实验室）
MINT: Mitigating Hallucinations in Large Vision-Language Models via Token Reduction
School of Future Technology, Shanghai University（上海大学未来技术学院）；Institute of Artificial Intelligence, Shanghai University（上海大学人工智能研究院）；School of Computer Engineering（计算机工程学院）
Mitigating Object Hallucinations via Sentence-Level Early Intervention
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；The Chinese University of Hong Kong（香港中文大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
MHSA: A Lightweight Framework for Mitigating Hallucinations via Steered Attention in LVLMs
Tsinghua University（清华大学）；Tsinghua University, Tencent（清华大学腾讯）；Tencent（腾讯）；University of Science and Technology Beijing（北京科技大学）；University of Macau（澳门大学）
Prescribing the Right Remedy: Mitigating Hallucinations in Large Vision-Language Models via Targeted Instruction Tuning
Beijing Key Lab of Traffic Data Analysis and Mining（北京交通大数据分析与挖掘重点实验室）；Beijing Jiaotong University（北京交通大学）；School of Information Technology and Management（信息科学技术学院）
SeeMe: Mitigating Hallucinations in Large Vision-Language Models through Effective Visual Token Engineering
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（北京大学计算机科学学院多媒体信息处理国家重点实验室）；University of Pennsylvania（宾夕法尼亚大学）；University of Electronic Science and Technology of China（电子科技大学）；Nanyang Technological University（南洋理工大学）；Tsinghua University（清华大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；De Artificial Intelligence Lab（德人工智能实验室）
Mitigating Entangled Steering in Large Vision-Language Models for Hallucination Reduction
Ministry of Education Key Laboratory of Intelligent Networks and Network Security（教育部智能网络与网络安全重点实验室）；Centre for Frontier AI Research, Institute of High Performance Computing, Agency for Science, Technology and Research（前沿人工智能研究中心，高性能计算研究所，科技研究局）
Look Clearly Before Answering: Mitigating Hallucinations in LVLMs via Saliency-Driven Perceptual Realignment
Xidian University（西安电子科技大学）；Tsinghua University（清华大学）；Shanghai Road Transport Development Center（上海市道路运输发展中心）；Hunan Institute of Advanced Technology（湖南先进技术研究院）
SAVE: Sparse Autoencoder-Driven Visual Information Enhancement for Mitigating Object Hallucination
Department of Electrical and Computer Engineering, Seoul National University（电子与计算机工程系，首尔国立大学）；Daegu Gyeongbuk Institute of Science and Technology（大邱庆州科学技术院）；IPAI, AIIS, ASRI, INMC, and ISRC, Seoul National University（IPAI、AIIS、ASRI、INMC 和 ISRC，首尔国立大学）
Prefill-Time Intervention for Mitigating Hallucination in Large Vision-Language Models
University of Science and Technology of China（中国科学技术大学）；Shanghai Advanced Research Institute, Chinese Academy of Sciences（上海先进研究院，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）
