arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.22483cs.CLcs.AI
收藏

提示优化在使用语言模型进行医疗笔记错误检测中的重要性

Importance of Prompt Optimisation for Error Detection in Medical Notes Using Language Models

University of St Andrews(圣安德鲁大学);Canon Medical Research Europe Ltd.(日本Canon医疗研究欧洲有限公司)

Craig Myles, Patrick Schrempf, David Harris-Birtill

AI总结:

本文通过提示优化提升语言模型在医疗笔记错误检测中的准确性,达到接近医生水平并超越MEDEC基准数据集表现。

摘要:

医疗文本中的错误可能导致延误甚至导致患者接受错误的治疗。最近,语言模型在自动检测医疗文本错误方面的能力展现出了潜力,这种能力有机会显著改善医疗系统。在本文中,我们探讨了在应用于错误检测任务时,提示优化对小型和大型语言模型的重要性。我们进行了严格的大规模语言模型和开源语言模型的实验和分析。我们证明了自动提示优化与遗传-帕累托(GEPA)相比,GPT-5的基线准确率从0.669提高到0.785,Qwen3-32B的基线准确率从0.578提高到0.690,接近医生的性能,并在MEDEC基准数据集上实现了最先进的性能。代码可在GitHub上获得:https://github.com/CraigMyles/clinical-note-error-detection

英文摘要:

Errors in medical text can cause delays or even result in incorrect treatment for patients. Recently, language models have shown promise in their ability to automatically detect errors in medical text, an ability that has the opportunity to significantly benefit healthcare systems. In this paper, we explore the importance of prompt optimisation for small and large language models when applied to the task of error detection. We perform rigorous experiments and analysis across frontier language models and open-source language models. We show that automatic prompt optimisation with Genetic-Pareto (GEPA) improves error detection over the baseline accuracy performance from 0.669 to 0.785 with GPT-5 and 0.578 to 0.690 with Qwen3-32B, approaching the performance of medical doctors and achieving state-of-the-art performance on the MEDEC benchmark dataset. Code available on GitHub: https://github.com/CraigMyles/clinical-note-error-detection

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑