arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.07319cs.CLcs.AI
收藏

超越准确性:对幻觉医疗建议的风险敏感评估

Beyond Accuracy: Risk-Sensitive Evaluation of Hallucinated Medical Advice

ASU(亚利桑那州立大学)

Savan Doshi

AI总结:

本文提出了一种风险敏感的评估框架,用于评估医疗建议中的幻觉风险,通过量化风险性语言来识别高风险、低支撑的失败模式。

摘要:

大型语言模型日益被用于患者面向的医学问答任务,其中幻觉输出可能在潜在危害上差异很大。然而,现有的幻觉标准和评估指标主要关注事实正确性,将所有错误视为同等严重。这掩盖了临床相关失败模式,特别是在模型生成未经支持但具有行动性的医疗语言时。我们提出了一种风险敏感的评估框架,通过风险性语言的存在来量化幻觉,包括治疗指示、禁忌、紧急提示以及高风险药物的提及。而不是评估临床正确性,我们的方法评估如果采取幻觉内容的潜在影响。我们进一步将风险评分与相关性度量相结合,以识别高风险、低支撑的失败。我们应用此框架对三个指令微调的语言模型使用受控的患者面向提示进行测试。我们的结果表明,表面行为相似的模型表现出显著不同的风险特征,并且标准评估指标无法捕捉这些区别。这些发现强调了在幻觉评估中纳入风险敏感性的重要性,并表明评估有效性在任务和提示设计上至关重要。

英文摘要:

Large language models are increasingly being used in patient-facing medical question answering, where hallucinated outputs can vary widely in potential harm. However, existing hallucination standards and evaluation metrics focus primarily on factual correctness, treating all errors as equally severe. This obscures clinically relevant failure modes, particularly when models generate unsupported but actionable medical language. We propose a risk-sensitive evaluation framework that quantifies hallucinations through the presence of risk-bearing language, including treatment directives, contraindications, urgency cues, and mentions of high-risk medications. Rather than assessing clinical correctness, our approach evaluates the potential impact of hallucinated content if acted upon. We further combine risk scoring with a relevance measure to identify high-risk, low-grounding failures. We apply this framework to three instruction-tuned language models using controlled patient-facing prompts designed as safety stress tests. Our results show that models with similar surface-level behavior exhibit substantially different risk profiles and that standard evaluation metrics fail to capture these distinctions. These findings highlight the importance of incorporating risk sensitivity into hallucination evaluation and suggest that evaluation validity is critically dependent on task and prompt design.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑