Measuring the Measurers: Quality Evaluation of Hallucination Benchmarks for Large Vision-Language Models
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, and also with University of Chinese Academy of Sciences（人工智能安全国家重点实验室，计算技术研究所，中国科学院，以及中国科学院大学）
Bei Yan, Jie Zhang, Zheng Yuan, Shiguang Shan, Xilin Chen
Despite the outstanding performance in multimodal tasks, Large Vision-Language Models (LVLMs) have been plagued by the issue of hallucination, i.e., generating content that is inconsistent with the corresponding visual inputs. While previous works have proposed various benchmarks to evaluate this issue, the quality of these evaluations remains unverified. We observe that some of these benchmarks may produce inconsistent evaluation results across repeated tests or fail to align with human evaluation. To address this, we propose a Hallucination benchmark Quality Measurement framework (HQM), which leverages specific indicators to assess both reliability and validity. Our empirical analysis using HQM reveals and pinpoints potential evaluation issues in existing benchmarks, exposing a critical gap in current hallucination evaluation. To bridge this gap, we propose HQH, a High-Quality Hallucination benchmark, which demonstrates superior reliability and validity under HQM, serving as a credible evaluation tool. Our large-scale evaluation of popular LVLMs on HQH reveals severe hallucination problems, which occur not only in the models' main answer to a question but also in additional analysis. This highlights the necessity for future model improvements to effectively mitigate hallucinations and reduce the associated security risks in real-world applications. Our benchmark is publicly available at https://github.com/HQHBench/HQHBench.
From Flat Facts to Sharp Hallucinations: Detecting Stubborn Errors via Gradient Sensitivity
School of Intelligent Manufacturing Ecosystem, Xi’an Jiaotong-Liverpool University, People’s Republic of China（智能制造生态系统学院，西安交通大学-利物浦大学，中华人民共和国）；Department of Computer Science, University of Liverpool, United Kingdom（计算机科学系，利物浦大学，英国）；Faculty of Engineering and Technology, Sunway University, Malaysia（工程与技术学院，Sunway大学，马来西亚）；School of Robotics, Xi’an Jiaotong-Liverpool University, People’s Republic of China（机器人学院，西安交通大学-利物浦大学，中华人民共和国）
What Color Is the Text? A Benchmark for Hallucination Induced by Image-Embedded Prompt
SKLCCSE, Institute of Artificial Intelligence, Beihang University, Beijing, China（软件学院、人工智能研究院、北航、北京、中国）；Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University, Beijing, China（未来区块链与隐私计算先进创新中心、北航、北京、中国）
A2RBench: An Automatic Paradigm for Formally Verifiable Abstract Reasoning Benchmark Generation
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education（教育部多媒体可信感知与高效计算重点实验室）；Institute of Artificial Intelligence（人工智能研究院）
PanCanBench: A Comprehensive Benchmark for Evaluating Large Language Models in Pancreatic Oncology
Department of Biostatistics, University of Washington（华盛顿大学生物统计学系）；Clinical Research Division, Fred Hutch Cancer Center（Fred Hutch癌症中心临床研究部）；Division of Hematology and Oncology, Department of Medicine, University of Washington（华盛顿大学医学系血液学与肿瘤学分会）；Allen Institute for AI（Allen人工智能研究所）；Department of Computer Science and Engineering, University of Washington（华盛顿大学计算机科学与工程系）；Public Health Sciences, Biostatistics, Fred Hutchinson Cancer Center（Fred Hutchinson癌症中心公共卫生科学与生物统计学）
URAG: A Benchmark for Uncertainty Quantification in Retrieval-Augmented Large Language Models
Uppsala University（乌普萨拉大学）；University of Science, VNU-HCM（VNU-HCM大学）；Indiana University（印第安纳大学）；FPT Software, AI Center（FPT软件人工智能中心）；The Pennsylvania State University（宾夕法尼亚州立大学）；VNU University of Engineering（VNU工程大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
