arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.04573cs.CL
收藏

给我一些难题:面向临床问答的合成数据生成

Give me Some Hard Questions: Synthetic Data Generation for Clinical QA

Johns Hopkins University(约翰斯·霍普金斯大学);Thomson Reuters(汤森路透);Optum

Fan Bai, Keith Harrigian, Joel Stremmel, Hamid Hassanzadeh, Ardavan Saeedi, Mark Dredze

AI总结:

针对临床问答标注数据稀缺且简单问题过多的问题,本文提出两种零样本LLM提示策略生成更难的合成问答,并在两个数据集上证明其可提升微调效果。

摘要:

临床问答(QA)系统使医生能够从电子健康记录(EHRs)中快速获取患者信息。然而,训练这些系统需要大量标注数据,而由于所需的专业知识以及临床数据相关的隐私顾虑,这类数据十分有限。本文探索在零样本设置下使用大型语言模型(LLMs)生成临床QA数据。我们发现,朴素提示往往会产生简单问题,无法反映临床场景的复杂性。为解决这一问题,我们提出两种提示策略:1)指示模型生成与输入上下文不重叠的问题;2)使用预定义模式总结输入记录,以为问题生成提供支架。在两个临床QA数据集上的实验表明,我们的方法生成了更具挑战性的问题,并相较基线显著提升了微调性能。我们比较了合成数据与黄金数据,发现二者在训练效果上存在差距,该差距源于合成生成答案的质量。

英文摘要:

Clinical Question Answering (QA) systems enable doctors to quickly access patient information from electronic health records (EHRs). However, training these systems requires significant annotated data, which is limited due to the expertise needed and the privacy concerns associated with clinical data. This paper explores generating Clinical QA data using large language models (LLMs) in a zero-shot setting. We find that naive prompting often results in easy questions that do not reflect the complexity of clinical scenarios. To address this, we propose two prompting strategies: 1) instructing the model to generate questions that do not overlap with the input context, and 2) summarizing the input record using a predefined schema to scaffold question generation. Experiments on two Clinical QA datasets demonstrate that our method generates more challenging questions, significantly improving fine-tuning performance over baselines. We compare synthetic and gold data and find a gap between their training efficacy resulting from the quality of synthetically generated answers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑