超越字面总结：重新定义医疗SOAP笔记评估中的幻觉
Beyond Literal Summarization: Redefining Hallucination for Medical SOAP Note Evaluation
AI总结：
本文提出通过校准提示和基于医学本体的检索，重新定义医疗SOAP笔记评估中的幻觉，发现传统评估方法高估了幻觉率，影响模型评估。
摘要：
评估大型语言模型（LLMs）在临床文档任务如SOAP笔记生成中的表现仍具挑战性。与标准总结不同，这些任务需要临床抽象、非正式语言的规范化以及医学基础推断。然而，现有的评估方法，包括自动化指标和LLM作为评判框架，依赖于词汇忠实度，通常将未在转录中明确出现的信息标记为幻觉。我们发现，此类方法系统性地将临床有效的输出标记为错误，提高幻觉率并扭曲模型评估。我们的分析表明，许多被标记为幻觉的内容对应于合法的临床转换，包括同义词映射、检查发现的抽象、诊断推断和符合指南的护理计划。通过校准提示和基于医学本体的检索对齐评估标准，我们观察到显著的成果变化。在词汇评估制度下，平均幻觉率为35%，严重惩罚有效推理。在推理意识评估下，此数值降至9%，剩余案例反映真正的安全问题。这些发现表明，当前评估实践过度惩罚有效临床推理，可能衡量的是评估设计的产物而非真实错误，凸显了在高上下文领域如医学中需要医学导向评估的必要性。
英文摘要：
Evaluating large language models (LLMs) for clinical documentation tasks such as SOAP note generation remains challenging. Unlike standard summarization, these tasks require clinical abstraction, normalization of colloquial language, and medically grounded inference. However, prevailing evaluation methods including automated metrics and LLM as judge frameworks rely on lexical faithfulness, often labeling any information not explicitly present in the transcript as hallucination. We show that such approaches systematically misclassify clinically valid outputs as errors, inflating hallucination rates and distorting model assessment. Our analysis reveals that many flagged hallucinations correspond to legitimate clinical transformations, including synonym mapping, abstraction of examination findings, diagnostic inference, and guideline consistent care planning. By aligning evaluation criteria with clinical reasoning through calibrated prompting and retrieval grounded in medical ontologies we observe a significant shift in outcomes. Under a lexical evaluation regime, the mean hallucination rate is 35%, heavily penalizing valid reasoning. With inference aware evaluation, this drops to 9%, with remaining cases reflecting genuine safety concerns. These findings suggest that current evaluation practices over penalize valid clinical reasoning and may measure artifacts of evaluation design rather than true errors, underscoring the need for clinically informed evaluation in high context domains like medicine.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
