MedHalu：大型语言模型在医疗查询中生成幻觉的研究
MedHalu: Hallucinations in Responses to Healthcare Queries by Large Language Models
AI总结：
MedHalu研究了LLM在医疗查询中生成幻觉的问题，提出MedHalu基准和MedHaluDetect框架，发现LLM在检测医学幻觉方面表现不佳，提出专家在环方法提升检测效果。
摘要：
大型语言模型（LLMs）正在开始补充传统的信息检索机制，如网络搜索。LLM驱动的聊天机器人如ChatGPT在一般公众中日益突出。AI聊天机器人也在越来越多地在社交媒体平台上生成内容。然而，LLMs也容易产生幻觉，生成看似合理但事实错误或伪造的信息。当普通大众开始寻求关于敏感问题如医疗保健的信息时，这成为了一个关键问题。现有在医学领域LLM幻觉方面的研究主要集中在通过标准化的医学考试问题测试LLM的医学知识，这些问题通常定义明确且答案明确。然而，这些方法可能无法完全捕捉LLMs在与患者进行真实世界互动时的表现。本研究进行了一项开创性的研究，探讨LLM生成对真实世界医疗查询的回应中的幻觉现象。我们介绍了MedHalu，一个新颖的医疗幻觉基准，包含多样的健康相关主题和LLM生成的幻觉回应，具有详细的幻觉类型和文本片段注释。我们还提出了MedHaluDetect，一个全面的框架，用于评估LLMs检测幻觉的能力。此外，我们研究了三组人对医学幻觉的易感性——医疗专家、LLMs和普通公众。值得注意的是，LLMs在检测医学幻觉方面显著表现不佳，有时甚至不如普通公众。为了提高幻觉检测能力，我们提出了一种专家在环的方法，将专家推理整合到LLM输入中，显著提高了所有LLM的幻觉检测能力，包括对GPT-4的宏F1指标提升了6.3%。我们的代码和数据集可在https://netsys.surrey.ac.uk/datasets/medhalu/上获得。
英文摘要：
Large language models (LLMs) are starting to complement traditional information seeking mechanisms such as web search. LLM-powered chatbots like ChatGPT are gaining prominence among the general public. AI chatbots are also increasingly producing content on social media platforms. However, LLMs are also prone to hallucinations, generating plausible yet factually incorrect or fabricated information. This becomes a critical problem when laypeople start seeking information about sensitive issues such as healthcare. Existing works in LLM hallucinations in the medical domain mainly focus on testing the medical knowledge of LLMs through standardized medical exam questions which are often well-defined and clear-cut with definitive answers. However, these approaches may not fully capture how these LLMs perform during real-world interactions with patients. This work conducts a pioneering study on hallucinations in LLM-generated responses to real-world healthcare queries from patients.We introduce MedHalu, a novel medical hallucination benchmark featuring diverse health-related topics and hallucinated responses from LLMs, with detailed annotation of the hallucination types and text spans. We also propose MedHaluDetect, a comprehensive framework for evaluating LLMs' abilities to detect hallucinations. Furthermore, we study the vulnerability to medical hallucinations among three groups -- medical experts, LLMs, and laypeople. Notably, LLMs significantly underperform human experts and, in some cases, even laypeople in detecting medical hallucinations. To improve hallucination detection, we propose an expert-in-the-loop approach that integrates expert reasoning into LLM inputs, significantly improving hallucination detection for all LLMs, including a 6.3% macro-F1 improvement for GPT-4. Our code and dataset are available at https://netsys.surrey.ac.uk/datasets/medhalu/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
