The rapid development of Large Multimodal Models (LMMs) has significantly advanced multimodal understanding by harnessing the language abilities of Large Language Models (LLMs) and integrating modality-specific encoders. However, LMMs are plagued by hallucinations that limit their reliability and adoption. While traditional methods to detect and mitigate these hallucinations often involve costly training or rely heavily on external models, recent approaches utilizing internal model features present a promising alternative. In this paper, we critically assess the limitations of the state-of-the-art training-free technique, the logit lens, in handling generalized visual hallucinations. We introduce ContextualLens, a refined method that leverages contextual token embeddings from middle layers of LMMs. This approach significantly improves hallucination detection and grounding across diverse categories, including actions and OCR, while also excelling in tasks requiring contextual understanding, such as spatial relations and attribute comparison. Our novel grounding technique yields highly precise bounding boxes, facilitating a transition from Zero-Shot Object Segmentation to Grounded Visual Question Answering. Our contributions pave the way for more reliable and interpretable multimodal models.
BEAF: Observing BEfore-AFter Changes to Evaluate Hallucination in Vision-language Models
POSTECH（浦项科技大学）；Grad. School of AI, POSTECH（浦项科技大学人工智能学院）；Institute for Convergence Research and Education in Advanced Technology, Yonsei University（延世大学尖端技术融合研究教育学院）
Tone Matters: The Impact of Linguistic Tone on Hallucination in VLMs
Department of Computer Science and Technology, Kean University（计算机科学与技术系，凯恩大学）；Department of Computer Science and Engineering, University of Notre Dame（计算机科学与工程系，圣母大学）
MedHallTune: An Instruction-Tuning Benchmark for Mitigating Medical Hallucination in Vision-Language Models
The Chinese University of Hong Kong（香港中文大学）；Institute of Medical Intelligence and XR, The Chinese University of Hong Kong（香港中文大学医学智能与XR研究所）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
Vision Language Models Map Logos to Text via Semantic Entanglement in the Visual Projector
University of California, Merced（加州大学梅尔德分校）；vivo Mobile Communication Co., Ltd.（vivo移动通信有限公司）；University of Queensland（昆士兰大学）；UCLA（加州大学洛杉矶分校）；University at Buffalo（布法罗大学）
Mitigating Hallucinations in Large Vision-Language Models with Internal Fact-based Contrastive Decoding
School of Future Technology, Shanghai University（上海大学未来技术学院）；Institute of Artificial Intelligence, Shanghai University（上海大学人工智能研究院）；School of Mechatronic Engineering and Automation（机电工程与自动化学院）
Direct Preference Optimization for Suppressing Hallucinated Prior Exams in Radiology Report Generation
Harvard University（哈佛大学）；Jawaharlal Institute of Postgraduate Medical Education & Research（贾瓦哈拉尔 postgraduate 医学教育与研究学院）；Johns Hopkins University（约翰斯·霍普金斯大学）
CDH-Bench: A Commonsense-Driven Hallucination Benchmark for Evaluating Visual Fidelity in Vision-Language Models
Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies, Institute of Cyberspace Security, School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen（广东省新型安全智能技术重点实验室，网络空间安全研究院，计算机科学与技术学院，哈尔滨工业大学（深圳））
Instruction-Aligned Visual Attention for Mitigating Hallucinations in Large Vision-Language Models
School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Cybersecurity, Northwestern Polytechnical University（西北工业大学网络空间安全学院）；Alibaba Group（阿里巴巴集团）；Zhejiang University of Technology（浙江工业大学）；Binjiang Institute of Artificial Intelligence（滨江人工智能研究院）
Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention
Xi’an Jiaotong University（西安交通大学）；National Engineering Laboratory for Big Data Analytics（国家大数据分析工程实验室）；Nanyang Technological University（南洋理工大学）；Lenovo Research（联想研究院）；University of Massachusetts Boston（马萨诸塞大学波士顿分校）；Zhejiang University of Technology（浙江工业大学）
Leveraging NTPs for Efficient Hallucination Detection in VLMs
Department of Data and Decision Science, Technion - Israel Institute of Technology（数据与决策科学系，技术学院-以色列理工学院）；Faculty of Computer and Information Science, Ben-Gurion University of the Negev（计算机与信息科学系，贝内-约尔根大学）
Understanding GUI Agent Localization Biases through Logit Sharpness
The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；The Hong Kong University of Science and Technology（香港科学与技术大学）；University of California, Merced（加州大学梅德福分校）；The University of Queensland（昆士兰大学）
Confabulations from ACL Publications (CAP): A Dataset for Scientific Hallucination Detection
Charles University, Prague（查尔斯大学，布拉格）；Independent Researcher（独立研究者）；University of Helsinki, Finland（赫尔辛基大学）；University of British Columbia, Vancouver, Canada（不列颠哥伦比亚大学，温哥华，加拿大）；Chosun University, South Korea（成均馆大学，韩国）
HalluSAE: Detecting Hallucinations in Large Language Models via Sparse Auto-Encoders
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University（北京未来区块链与隐私计算先进研究院，北京航空航天大学人工智能学院）；State Key Laboratory of Virtual Reality Technology and Systems, Beihang University（北京航空航天大学虚拟现实技术与系统国家重点实验室）；Beijing Academy of Blockchain and Edge Computing（北京区块链与边缘计算研究院）；Renmin University of China（中国人民大学）
Detection and Mitigation of Hallucination in Large Reasoning Models: A Mechanistic Perspective
Gaoling School of Artificial Intelligence（北京理工大学人工智能学院）；Engineering Research Center of Next-Generation Intelligent Search and Recommendation（下一代智能搜索与推荐工程研究中心）
