MedVH：面向医学场景大视觉语言模型幻觉的系统性评估
MedVH: Towards Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical Context
AI总结：
本文提出医学视觉幻觉测试基准MedVH，通过五个任务系统评估医学大视觉语言模型的幻觉问题，实验发现医学模型虽在标准任务表现良好但更易产生幻觉，强调其需兼顾知识整合与稳健推理。
摘要：
大视觉语言模型（LVLMs）最近在自然图像和文本数据的各类任务中取得了卓越性能，这激发了大量针对LVLMs微调和训练的研究。尽管它们取得了进展，但关于这些模型在较小数据集上微调后对幻觉的鲁棒性研究仍然很少。在本研究中，我们引入了一个新的基准数据集，即医学视觉幻觉测试（MedVH），用于评估领域特定LVLMs的幻觉情况。MedVH包含五个任务，用于评估医学场景下LVLMs的幻觉，其中包括对文本和视觉输入进行全面理解的任务，以及长文本响应生成任务。我们使用通用和医学LVLMs进行的大量实验表明，尽管医学LVLMs在标准医学任务上表现出有前景的性能，但它们特别容易产生幻觉，往往比通用模型更容易出现幻觉，这引发了人们对这些领域特定模型可靠性的重大担忧。为了使医学LVLMs在现实应用中真正有价值，它们不仅必须准确整合医学知识，还必须保持稳健的推理能力以防止幻觉。我们的工作为未来对这些研究的评估铺平了道路。
英文摘要：
Large Vision Language Models (LVLMs) have recently achieved superior performance in various tasks on natural image and text data, which inspires a large amount of studies for LVLMs fine-tuning and training. Despite their advancements, there has been scant research on the robustness of these models against hallucination when fine-tuned on smaller datasets. In this study, we introduce a new benchmark dataset, the Medical Visual Hallucination Test (MedVH), to evaluate the hallucination of domain-specific LVLMs. MedVH comprises five tasks to evaluate hallucinations in LVLMs within the medical context, which includes tasks for comprehensive understanding of textual and visual input, as well as long textual response generation. Our extensive experiments with both general and medical LVLMs reveal that, although medical LVLMs demonstrate promising performance on standard medical tasks, they are particularly susceptible to hallucinations, often more so than the general models, raising significant concerns about the reliability of these domain-specific models. For medical LVLMs to be truly valuable in real-world applications, they must not only accurately integrate medical knowledge but also maintain robust reasoning abilities to prevent hallucination. Our work paves the way for future evaluations of these studies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
