HEDGE：基于密集几何熵的幻觉估计用于视觉语言模型的视觉问答
HEDGE: Hallucination Estimation via Dense Geometric Entropy for VQA with Vision-Language Models
AI总结：
针对视觉语言模型（VLMs）在视觉问答中的幻觉问题，提出HEDGE框架，结合受控视觉扰动、语义聚类和鲁棒不确定性度量，在多个数据集和模型上验证了其有效性，为多模态可靠性评估提供基础并发布基准库。
摘要：
视觉语言模型（VLMs）支持开放式视觉问答，但仍易产生幻觉。我们提出HEDGE，一个统一的幻觉检测框架，结合受控视觉扰动、语义聚类和鲁棒不确定性度量。HEDGE将采样、失真合成、聚类（基于蕴含和嵌入）以及度量计算整合到可复现的管道中，适用于多种多模态架构。在VQA-RAD和KvasirVQA-x1数据集上，使用三种代表性VLMs（LLaVA-Med、Med-Gemma、Qwen2.5-VL）进行评估，揭示了清晰的架构和提示依赖趋势。幻觉检测能力在具有密集视觉标记化的统一融合模型（Qwen2.5-VL）中最高，在标记化受限的架构（Med-Gemma）中最低。基于嵌入的聚类直接应用于生成答案时往往产生更强的分离效果，而基于NLI的聚类对LLaVA-Med和较长的句子级响应仍具优势。在所有配置中，VASE度量始终提供最鲁棒的幻觉信号，尤其是与嵌入聚类和适度采样预算（n≈10-15）结合时。提示设计也很重要：简洁的标签式输出比语法受限的单句响应具有更清晰的语义结构。通过将幻觉检测构建为受采样规模、提示结构、模型架构和聚类策略共同影响的几何鲁棒性问题，HEDGE为评估多模态可靠性提供了原则性、计算感知的基础。hedge-bench PyPI库支持可复现和可扩展的基准测试，完整代码和实验资源可在https://github.com/Simula/HEDGE获取。
英文摘要：
Vision-language models (VLMs) enable open-ended visual question answering but remain prone to hallucinations. We present HEDGE, a unified framework for hallucination detection that combines controlled visual perturbations, semantic clustering, and robust uncertainty metrics. HEDGE integrates sampling, distortion synthesis, clustering (entailment- and embedding-based), and metric computation into a reproducible pipeline applicable across multimodal architectures. Evaluations on VQA-RAD and KvasirVQA-x1 with three representative VLMs (LLaVA-Med, Med-Gemma, Qwen2.5-VL) reveal clear architecture- and prompt-dependent trends. Hallucination detectability is highest for unified-fusion models with dense visual tokenization (Qwen2.5-VL) and lowest for architectures with restricted tokenization (Med-Gemma). Embedding-based clustering often yields stronger separation when applied directly to the generated answers, whereas NLI-based clustering remains advantageous for LLaVA-Med and for longer, sentence-level responses. Across configurations, the VASE metric consistently provides the most robust hallucination signal, especially when paired with embedding clustering and a moderate sampling budget (n ~ 10-15). Prompt design also matters: concise, label-style outputs offer clearer semantic structure than syntactically constrained one-sentence responses. By framing hallucination detection as a geometric robustness problem shaped jointly by sampling scale, prompt structure, model architecture, and clustering strategy, HEDGE provides a principled, compute-aware foundation for evaluating multimodal reliability. The hedge-bench PyPI library enables reproducible and extensible benchmarking, with full code and experimental resources available at https://github.com/Simula/HEDGE .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
