Large Vision-Language Models (LVLMs) have achieved remarkable performance on diverse vision-language tasks. However, LVLMs still suffer from hallucinations, generating text that contradicts the visual input. Existing research has primarily focused on mitigating object hallucinations, but often overlooks more complex relation hallucinations, particularly action relations involving interactions between objects. In this study, we empirically observe that the primary cause of action-relation hallucinations in LVLMs is the insufficient attention allocated to visual information. Thus, we propose a framework to locate action-relevant image regions and enhance the LVLM's attention to those regions. Specifically, we define the Action-Relation Sensitivity (ARS) score to identify attention heads that are most sensitive to action-relation changes, thereby localizing action-relevant image regions that contain key visual cues. Then, we propose the Relation-aware Visual Enhancement (RVE) method to enhance the LVLM's attention to these action-relevant image regions. Extensive experiments demonstrate that, compared to existing baselines, our method achieves superior performance in mitigating action-relation hallucinations with negligible additional inference cost. Furthermore, it effectively generalizes to spatial-relation hallucinations and object hallucinations.
Attention Hijackers: Detect and Disentangle Attention Hijacking in LVLMs for Hallucination Mitigation
Center for Future Media, University of Electronic Science and Technology of China（电子科技大学未来媒体中心）；Southwestern University of Finance and Economics（西南财经大学）；Tongji University（同济大学）
HIME: Mitigating Object Hallucinations in LVLMs via Hallucination Insensitivity Model Editing
School of Information and Communication Technology, Griffith University, Australia（信息与通信技术学院，格里菲斯大学）；Data61, CSIRO, Australia（Data61，澳大利亚联邦科学与工业研究组织）；School of Engineering, Macquarie University, Sydney, Australia（工程学院，麦觉大学）；School of Information Systems, Queensland University of Technology, Australia（信息系统学院，昆士兰技术大学）
MHSA: A Lightweight Framework for Mitigating Hallucinations via Steered Attention in LVLMs
Tsinghua University（清华大学）；Tsinghua University, Tencent（清华大学腾讯）；Tencent（腾讯）；University of Science and Technology Beijing（北京科技大学）；University of Macau（澳门大学）
Revealing and Enhancing Core Visual Regions: Harnessing Internal Attention Dynamics for Hallucination Mitigation in LVLMs
School of Electronic Engineering, Xidian University（西安电子科技大学电子工程学院）；School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；College of Computer and Information, Hohai University（河海大学计算机与信息学院）；Institute for Infocomm Research, A*STAR（A*STAR信息与通信研究所）
Towards Interpretable Hallucination Analysis and Mitigation in LVLMs via Contrastive Neuron Steering
School of Electronic Engineering, Xidian University, Xi'an, China（西安电子科技大学电子工程学院）；School of Computer Science and Technology, Xidian University, Xi'an, China（西安电子科技大学计算机科学与技术学院）；College of Computer and Information, Hohai University, Nanjing, China（河海大学计算机与信息学院）；Institute for Infocomm Research, A*STAR, Singapore（新加坡资讯研究院）
Beyond Global Editing: Per-Instance Disentangled Subspaces for Training-Free Hallucination Mitigation in LVLMs
Macquarie University（麦考瑞大学）；York University（约克大学）；Northern Illinois University（北伊利诺伊大学）；University of Technology Sydney（悉尼科技大学）；North South University（北南大学）；Lancaster University（兰卡斯特大学）；Australian National University（澳大利亚国立大学）
Revealing Perception and Generation Dynamics in LVLMs: Mitigating Hallucinations via Validated Dominance Correction
School of Electronic Engineering, Xidian University（电子科技大学电子工程学院）；School of Computer Science and Technology, Xidian University（电子科技大学计算机科学与技术学院）；Hohai university（河海大学）；Institute for Infocomm Research (I 2 R), A*STAR（信息通信研究所（I2R））；School of Foreign Languages, Xidian University（电子科技大学外国语言学院）
Detecting and Evaluating Medical Hallucinations in Large Vision Language Models
Academy for Engineering and Technology, Fudan University（复旦大学工程与技术学院）；Tencent Youtu Lab（腾讯优图实验室）；Cognition and Intelligent Technology Laboratory（认知与智能技术实验室）
SHIELD: Suppressing Hallucinations In LVLM Encoders via Bias and Vulnerability Defense
Department of Electrical and Computer Engineering, Northeastern University（电气与计算机工程系，东北大学）；Khoury College of Computer Science, Northeastern University（计算机科学学院，东北大学）
VOPE: Revisiting Hallucination of Vision-Language Models in Voluntary Imagination Task
Key Laboratory of AI Safety of CAS, Institute of Computing Technology, Chinese Academy of Sciences (CAS)（中国科学院人工智能安全重点实验室，计算技术研究所，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Zhongguancun Academy（中关村学院）
Detecting Clinical Hallucinations in LVLMs via Counterfactual Visual Grounding Uncertainty
School of Intelligent Science and Technology, Nanjing University（南京大学智能科学与技术学院）；National Institute of Healthcare Data Science at Nanjing University（南京大学医疗数据科学国家研究院）；School of Biomedical Engineering, Nanjing University（南京大学生物医学工程学院）
CATCH: Complementary Adaptive Token-level Contrastive Decoding to Mitigate Hallucinations in LVLMs
Tsinghua University（清华大学）；Pengcheng Laboratory（鹏城实验室）；Carnegie Mellon University（卡内基梅隆大学）；University of California, Berkeley（加州大学伯克利分校）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）；Chang’an University（长安大学）
Beyond Visual Enhancement: Adaptive Multi-Context Steering to Mitigate LVLM Hallucinations
Shanghai Jiao Tong University（上海交通大学）；Peking University（北京大学）；Beijing University of Chemical Technology（北京化工大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；University of Science and Technology of China（中国科学技术大学）；Tsinghua University（清华大学）
Beyond the Global Scores: Fine-Grained Token Grounding as a Robust Detector of LVLM Hallucinations
Hanoi University of Science and Technology（河内科技大学）；Australian Institute for Machine Learning, University of Adelaide（澳大利亚机器学习研究所，阿德莱德大学）；Nanjing University（南京大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Hanoi-Amsterdam High School for the Gifted（河内阿姆斯特丹天才高中）
Cross-Modal Attention Calibration for LVLM Hallucination Mitigation
Sun Yat-sen University（中山大学）；The University of Hong Kong（香港大学）；Meituan（美团）；Inspur Database Technology（Inspur数据库技术）；Guilin University of Electronic Technology（桂林电子科技大学）；Shenzhen Loop Area Institute（深圳河套学院）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东大数据分析与处理重点实验室）
What Makes LVLMs Hallucinate Less? Unveiling the Architectural Factors Behind Hallucination Robustness
School of Computer Science, Engineering Research Center of Machine Learning and Industry Intelligence, Sichuan University（计算机科学学院，机器学习与产业智能工程研究中心，四川大学）；School of Computer and Information Engineering, Xiamen University of Technology（计算机与信息工程学院，厦门理工大学）；Department of Electrical and Computer Engineering, University of Hong Kong（电气与计算机工程系，香港大学）；College of Computing and Data Science, Nanyang Technological University（计算与数据科学学院，南洋理工大学）
