对语言模型在医学教科书中 hallucinations 的量化
Quantifying Hallucinations in Language Language Models on Medical Textbooks
AI总结：
本文研究了大型语言模型在医学教科书基础上的 hallucinations 发生频率及不同模型响应的差异，发现即使在高可信度响应下，LLaMA-70B-Instruct仍存在19.7%的hallucinations，且临床专家对模型响应的评估显示高一致性。
摘要：
Hallucinations，即大型语言模型倾向于提供事实性错误和无依据的主张，是自然语言处理中的严重问题，目前尚无有效解决方案。现有医学问答基准很少评估此行为相对于固定证据源。我们询问在教科书基础的问答中hallucinations发生频率以及不同模型对医学问答提示的响应差异。我们进行了两个实验，第一个实验确定在封闭源零样本提示下，知名开源大型语言模型（LLaMA-70B-Instruct）在医学问答中的hallucinations普遍性，第二个实验确定hallucinations发生率和临床专家对模型响应的偏好。我们观察到，在第一个实验中，尽管98.8%的提示响应获得最大可信度，LLaMA-70B-Instruct在19.7%的答案中hallucinated（95% CI 18.6到20.7）。在第二个实验中，不同模型中较低的hallucination率与较高的有用性评分（ρ=-0.71，p=0.058）相关。临床专家在实验1和2中分别表现出高一致性（二次加权κ=0.92）和（τ_b=0.06到0.18，κ=0.57到0.61）。我们的发现表明，在所有测试的尺度和架构中，当前大型语言模型仍不适合无监督的临床部署，且人类专家监督既是必要也是主导成本驱动因素。
英文摘要：
Hallucinations, the tendency for large language models to provide responses with factually incorrect and unsupported claims, is a serious problem within natural language processing for which we do not yet have an effective solution to mitigate against. Existing benchmarks for medical QA rarely evaluate this behavior against a fixed evidence source. We ask how often hallucinations occur on textbook-grounded QA and how responses to medical QA prompts vary across models. We conduct two experiments, the first experiment to determine the prevalence of hallucinations for a prominent open source large language model (LLaMA-70B-Instruct) in medical QA given closed-source zero-shot prompts, and the second experiment to determine the prevalence of hallucinations and clinician preference to model responses. We observed, in experiment one, with the passages provided, LLaMA-70B-Instruct hallucinated in 19.7\% of answers (95\% CI 18.6 to 20.7) even though 98.8\% of prompt responses received maximal plausibility, and observed in experiment two, across models, lower hallucination rates aligned with higher usefulness scores ($ρ=-0.71$, $p=0.058$). Clinicians produced high agreement (quadratic weighted $κ=0.92$) and ($τ_b=0.06$ to $0.18$, $κ=0.57$ to $0.61$) for experiments 1 and 2 respectively. Our findings indicate that, across all scales and architectures tested, current large language models remain unfit for unsupervised clinical deployment, and that human expert oversight is both necessary and the dominant cost driver.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
