arXiv 2509.203792025-11-14cs.CVcs.CLcs.LG

利用NTP实现视觉语言模型中的高效幻觉检测

Leveraging NTPs for Efficient Hallucination Detection in VLMs

Department of Data and Decision Science, Technion - Israel Institute of Technology（数据与决策科学系，技术学院-以色列理工学院）；Faculty of Computer and Information Science, Ben-Gurion University of the Negev（计算机与信息科学系，贝内-约尔根大学）

Ofir Azachi, Kfir Eliyahu, Eyal El Ani, Rom Himelstein, Roi Reichart, Yuval Pinter, Nitay Calderon

备注：: Accepted to The First Workshop on Confabulation, Hallucinations, & Overgeneration in Multilingual & Precision-critical Setting - AACL-IJCNLP2025

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对VLM幻觉检测现有方法计算成本高、延迟大的问题，本文提出基于NTP信号训练轻量ML模型的实时检测方法，通过构建标注数据集验证其可达到与强VLM相当的检测性能，融合多类信号还可进一步提升效果。

摘要：

视觉语言模型（VLMs）的幻觉指视觉内容与生成文本之间的错位问题，会削弱VLMs的可靠性。检测幻觉的一类常见方法是使用同一VLM或其他VLM评估生成输出，该过程计算量极大，还会增加模型延迟。本文探索了一种高效的实时幻觉检测方法，基于VLM的下一词元概率（NTPs）信号训练传统机器学习模型。NTPs可直接量化模型不确定性，我们假设高不确定性（即低NTP值）与幻觉存在强关联。为验证该假设，我们构建了一个包含1400条人工标注语句的数据集，这些语句均来自VLM生成内容，每条都标注了是否存在幻觉，我们用该数据集测试了基于NTP的轻量方法。结果表明，基于NTP的特征是有效的幻觉预测因子，能让快速简单的机器学习模型达到与强VLM相当的性能。此外，将仅把生成文本反馈给VLM计算得到的语言NTP补充到原有NTP中，可提升幻觉检测性能。最后，将VLM输出的幻觉预测分数整合到基于NTP的模型中，能取得比单独使用VLM或NTP更优的效果。我们希望本研究能为提升VLM可靠性的简单轻量解决方案铺平道路。

英文摘要：

Hallucinations of vision-language models (VLMs), which are misalignments between visual content and generated text, undermine the reliability of VLMs. One common approach for detecting them employs the same VLM, or a different one, to assess generated outputs. This process is computationally intensive and increases model latency. In this paper, we explore an efficient on-the-fly method for hallucination detection by training traditional ML models over signals based on the VLM's next-token probabilities (NTPs). NTPs provide a direct quantification of model uncertainty. We hypothesize that high uncertainty (i.e., a low NTP value) is strongly associated with hallucinations. To test this, we introduce a dataset of 1,400 human-annotated statements derived from VLM-generated content, each labeled as hallucinated or not, and use it to test our NTP-based lightweight method. Our results demonstrate that NTP-based features are valuable predictors of hallucinations, enabling fast and simple ML models to achieve performance comparable to that of strong VLMs. Furthermore, augmenting these NTPs with linguistic NTPs, computed by feeding only the generated text back into the VLM, enhances hallucination detection performance. Finally, integrating hallucination prediction scores from VLMs into the NTP-based models led to better performance than using either VLMs or NTPs alone. We hope this study paves the way for simple, lightweight solutions that enhance the reliability of VLMs.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。