Hallu-PI: Evaluating Hallucination in Multi-modal Large Language Models within Perturbed Inputs
National Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Meituan（美团）；Zhejiang-Singapore Innovation and AI Joint Research Lab, Zhejiang University（浙江大学浙新创新与AI联合研究实验室）
Peng Ding, Jingyu Wu, Jun Kuang, Dan Ma, Xuezhi Cao, Xunliang Cai, Shi Chen, Jiajun Chen, Shujian Huang
备注：
Acccepted by ACM MM 2024, 14 pages, 11 figures, 9 tables
Multi-modal Large Language Models (MLLMs) have demonstrated remarkable performance on various visual-language understanding and generation tasks. However, MLLMs occasionally generate content inconsistent with the given images, which is known as "hallucination". Prior works primarily center on evaluating hallucination using standard, unperturbed benchmarks, which overlook the prevalent occurrence of perturbed inputs in real-world scenarios-such as image cropping or blurring-that are critical for a comprehensive assessment of MLLMs' hallucination. In this paper, to bridge this gap, we propose Hallu-PI, the first benchmark designed to evaluate Hallucination in MLLMs within Perturbed Inputs. Specifically, Hallu-PI consists of seven perturbed scenarios, containing 1,260 perturbed images from 11 object types. Each image is accompanied by detailed annotations, which include fine-grained hallucination types, such as existence, attribute, and relation. We equip these annotations with a rich set of questions, making Hallu-PI suitable for both discriminative and generative tasks. Extensive experiments on 12 mainstream MLLMs, such as GPT-4V and Gemini-Pro Vision, demonstrate that these models exhibit significant hallucinations on Hallu-PI, which is not observed in unperturbed scenarios. Furthermore, our research reveals a severe bias in MLLMs' ability to handle different types of hallucinations. We also design two baselines specifically for perturbed scenarios, namely Perturbed-Reminder and Perturbed-ICL. We hope that our study will bring researchers' attention to the limitations of MLLMs when dealing with perturbed inputs, and spur further investigations to address this issue. Our code and datasets are publicly available at https://github.com/NJUNLP/Hallu-PI.
SHALE: A Scalable Benchmark for Fine-grained Hallucination Evaluation in LVLMs
Key Laboratory of AI Safety of CAS, Institute of Computing Technology, Chinese Academy of Sciences (CAS)（中国科学院计算技术研究所人工智能安全重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Trustworthy Technology and Engineering Laboratory, Huawei（华为可信技术与工程实验室）
Alleviating Hallucinations in Large Vision-Language Models through Hallucination-Induced Optimization
Southwestern University of Finance and Economics（西南财经大学）；Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China（电子科技大学深圳高等研究院）；Center for Future Media, University of Electronic Science and Technology of China（电子科技大学未来媒体中心）；Tongji University（同济大学）
Detecting and Evaluating Medical Hallucinations in Large Vision Language Models
Academy for Engineering and Technology, Fudan University（复旦大学工程与技术学院）；Tencent Youtu Lab（腾讯优图实验室）；Cognition and Intelligent Technology Laboratory（认知与智能技术实验室）
There are many factors affecting visual face recognition, such as low resolution images, aging, illumination and pose variance, etc. One of the most important problem is low resolution face images which can result in bad performance on face recognition. Most of the general face recognition algorithms usually assume a sufficient resolution for the face images. However, in practice many applications often do not have sufficient image resolutions. The modern face hallucination models demonstrate reasonable performance to reconstruct high-resolution images from its corresponding low resolution ima…
MHSA: A Lightweight Framework for Mitigating Hallucinations via Steered Attention in LVLMs
Tsinghua University（清华大学）；Tsinghua University, Tencent（清华大学腾讯）；Tencent（腾讯）；University of Science and Technology Beijing（北京科技大学）；University of Macau（澳门大学）
DHCP: Detecting Hallucinations by Cross-modal Attention Pattern in Large Vision-Language Models
Tsinghua University, Tencent（清华大学、腾讯）；Tencent（腾讯）；Tencent, University of Macau（腾讯、澳门大学）；University of Science and Technology Beijing（北京科技大学）；Tsinghua University（清华大学）
Prescribing the Right Remedy: Mitigating Hallucinations in Large Vision-Language Models via Targeted Instruction Tuning
Beijing Key Lab of Traffic Data Analysis and Mining（北京交通大数据分析与挖掘重点实验室）；Beijing Jiaotong University（北京交通大学）；School of Information Technology and Management（信息科学技术学院）
HalluGen: Synthesizing Realistic and Controllable Hallucinations for Evaluating Image Restoration
Hawkes Institute, UCL（UCL哈维斯研究所）；Dept. of Medical Physics and Biomedical Engineering, UCL（UCL医学物理与生物医学工程系）；Dept. of Computer Science, UCL（UCL计算机科学系）；Centre for AI, DS&AI, AstraZeneca, UK（阿斯利康AI中心）
Mitigating Hallucinations in Large Vision-Language Models by Self-Injecting Hallucinations
Beijing Key Laboratory of Super Intelligent Security of Multi-Modal Information, CASIA（北京多模态信息超级智能安全重点实验室，中国科学院自动化所）；State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA（多模态人工智能系统国家重点实验室，中国科学院自动化所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Hello Group（Hello集团）；Nanchang Hangkong University（南昌航空大学）；The University of Hong Kong（香港大学）；School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）
Despite continuously improving performance, contemporary image captioning models are prone to "hallucinating" objects that are not actually in a scene. One problem is that standard metrics only measure similarity to ground truth captions and may not fully capture image relevance. In this work, we propose a new image relevance metric to evaluate current models with veridical visual labels and assess their rate of object hallucination. We analyze how captioning model architectures and learning objectives contribute to object hallucination, explore when hallucination is likely due to image miscla…
The Hong Kong University of Science and Technology（香港科技大学）；NVIDIA（英伟达）
Large-scale vision-language pre-trained (VLP) models are prone to hallucinate non-existent visual objects when generating text based on visual information. In this paper, we systematically study the object hallucination problem from three aspects. First, we examine recent state-of-the-art VLP models, showing that they still hallucinate frequently, and models achieving better scores on standard metrics (e.g., CIDEr) could be more unfaithful. Second, we investigate how different types of image encoding in VLP influence hallucination, including region-based, grid-based, and patch-based. Surprisin…
What Makes LVLMs Hallucinate Less? Unveiling the Architectural Factors Behind Hallucination Robustness
School of Computer Science, Engineering Research Center of Machine Learning and Industry Intelligence, Sichuan University（计算机科学学院，机器学习与产业智能工程研究中心，四川大学）；School of Computer and Information Engineering, Xiamen University of Technology（计算机与信息工程学院，厦门理工大学）；Department of Electrical and Computer Engineering, University of Hong Kong（电气与计算机工程系，香港大学）；College of Computing and Data Science, Nanyang Technological University（计算与数据科学学院，南洋理工大学）
Mitigating Entangled Steering in Large Vision-Language Models for Hallucination Reduction
Ministry of Education Key Laboratory of Intelligent Networks and Network Security（教育部智能网络与网络安全重点实验室）；Centre for Frontier AI Research, Institute of High Performance Computing, Agency for Science, Technology and Research（前沿人工智能研究中心，高性能计算研究所，科技研究局）
HalluShift++: Bridging Language and Vision through Internal Representation Shifts for Hierarchical Hallucinations in MLLMs
Netaji Subhash Engineering College (NSEC)（奈尔贾伊·萨布哈工程学院）；TCG Crest；Electronics and Communication Sciences Unit (ECSU)（电子与通信科学单位）；Indian Statistical Institute（印度统计研究所）
Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images
Provable Responsible AI and Data Analytics (PRADA) Lab（可证明负责任的人工智能与数据分析实验室）；King Abdullah University of Science and Technology（国王阿卜杜勒阿齐兹大学）；University of Electronic Science and Technology of China（中国电子科学技术大学）；University of Copenhagen（哥本哈根大学）
Evaluation and Analysis of Hallucination in Large Vision-Language Models
Beijing Jiaotong University（北京交通大学）；Xi’an Jiaotong University（西安交通大学）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Shandong University（山东大学）
