arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2311.02271cs.CLcs.AI

FaMeSumm:调查并提升医疗摘要的忠实度

FaMeSumm: Investigating and Improving Faithfulness of Medical Summarization

The Pennsylvania State University(宾夕法尼亚州立大学);Children’s Hospital Affiliated of Zhengzhou University(郑州大学附属儿童医院);L3S Research Center(L3S研究中心)

Nan Zhang, Yusen Zhang, Wu Guo, Prasenjit Mitra, Rui Zhang

AI总结:

针对医疗摘要忠实度不足的问题,提出基于医疗知识微调预训练模型的FaMeSumm框架,通过对比学习与术语上下文约束提升忠实度,在多语言多数据集上实现优于主流模型的效果。

摘要:

医疗文本的摘要应当具备忠实性,即与源输入保持一致且符合事实,这对医疗保健领域的安全性与效率而言是一个重要但研究不足的课题。本文针对广泛的医疗摘要任务,调查并提升摘要的忠实度。我们的调查显示,当前的摘要模型针对医疗输入文本生成的输出往往不具备忠实性。随后我们提出FaMeSumm框架,该框架基于医疗知识对预训练语言模型进行微调,以此提升忠实度。FaMeSumm在精心设计的忠实与不忠实摘要集合上开展对比学习,同时融入医疗术语及其上下文,以促进医疗术语的忠实生成。我们在两种语言的三个数据集上开展了全面实验:英文的健康问题摘要数据集与放射报告摘要数据集,以及中文的医患对话数据集。结果表明,FaMeSumm具备灵活性与有效性,相较于BART、T5、mT5和PEGASUS等主流语言模型实现了持续提升,在忠实度与通用质量的指标上取得了当前最优性能。由医生开展的人工评估也显示,FaMeSumm生成的输出更具忠实性。我们的代码可在https://github.com/psunlpgroup/FaMeSumm 获取。

英文摘要:

Summaries of medical text shall be faithful by being consistent and factual with source inputs, which is an important but understudied topic for safety and efficiency in healthcare. In this paper, we investigate and improve faithfulness in summarization on a broad range of medical summarization tasks. Our investigation reveals that current summarization models often produce unfaithful outputs for medical input text. We then introduce FaMeSumm, a framework to improve faithfulness by fine-tuning pre-trained language models based on medical knowledge. FaMeSumm performs contrastive learning on designed sets of faithful and unfaithful summaries, and it incorporates medical terms and their contexts to encourage faithful generation of medical terms. We conduct comprehensive experiments on three datasets in two languages: health question and radiology report summarization datasets in English, and a patient-doctor dialogue dataset in Chinese. Results demonstrate that FaMeSumm is flexible and effective by delivering consistent improvements over mainstream language models such as BART, T5, mT5, and PEGASUS, yielding state-of-the-art performances on metrics for faithfulness and general quality. Human evaluation by doctors also shows that FaMeSumm generates more faithful outputs. Our code is available at https://github.com/psunlpgroup/FaMeSumm .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑