一种以数据为中心的方法，利用大语言模型生成忠实且高质量的病人摘要
A Data-Centric Approach To Generate Faithful and High Quality Patient Summaries with Large Language Models
AI总结：
研究利用大语言模型根据医生笔记生成病人摘要，提出严格标注协议和公开幻觉数据集，证明在无幻觉数据上微调或提供无幻觉少样本示例能有效降低 Llama 2 和 GPT-4 的幻觉，并发现 GPT-4 自动幻觉检测优于常见基线。
摘要：
患者往往难以理解其住院情况，而医护人员又缺乏足够的资源来提供解释。在这项工作中，我们研究了大语言模型根据医生笔记生成病人摘要的潜力，并探讨了训练数据对生成摘要的忠实度和质量的影响。为此，我们发布了（i）一套针对医学文本中错误的严格标注协议，以及（ii）一个公开可用的数据集，包含对100份医生撰写和100份生成的摘要中幻觉的标注。我们展示了在无幻觉数据上微调能有效减少 Llama 2 每份摘要的幻觉（从2.60降至1.55），同时保留相关信息。当少样本示例无幻觉时，我们观察到 GPT-4 也有类似效果（从0.70降至0.40）。我们还使用无幻觉和改进的训练数据进行了定性评估。我们发现，常见的定量指标与忠实度和质量的相关性不高。最后，我们测试了 GPT-4 进行自动幻觉检测的能力，其表现明显优于常见基线。
英文摘要：
Patients often face difficulties in understanding their hospitalizations, while healthcare workers have limited resources to provide explanations. In this work, we investigate the potential of large language models to generate patient summaries based on doctors' notes and study the effect of training data on the faithfulness and quality of the generated summaries. To this end, we release (i) a rigorous labeling protocol for errors in medical texts and (ii) a publicly available dataset of annotated hallucinations in 100 doctor-written and 100 generated summaries. We show that fine-tuning on hallucination-free data effectively reduces hallucinations from 2.60 to 1.55 per summary for Llama 2, while preserving relevant information. We observe a similar effect on GPT-4 (0.70 to 0.40), when the few-shot examples are hallucination-free. We also conduct a qualitative evaluation using hallucination-free and improved training data. We find that common quantitative metrics do not correlate well with faithfulness and quality. Finally, we test GPT-4 for automatic hallucination detection, which clearly outperforms common baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
