迭代提示精炼用于放射肿瘤学症状提取：基于教师-学生大语言模型
Iterative Prompt Refinement for Radiation Oncology Symptom Extraction Using Teacher-Student Large Language Models
AI总结：
本研究提出教师-学生大语言模型架构，通过GPT-4迭代精炼提示来改进Mixtral从前列腺癌放疗临床笔记中提取症状的性能，显著提升了准确率、精确率、召回率和F1分数。
摘要：
本研究引入了一种新颖的教师-学生架构，利用大语言模型（LLMs）来改进从临床笔记中提取前列腺癌放疗症状的效果。学生模型Mixtral首先提取症状，随后教师模型GPT-4根据Mixtral的表现对提示进行精炼。这一迭代过程涉及294条单症状临床笔记，涵盖12种症状，每个周期最多进行16轮精炼。结果显示，从单症状和多症状笔记中提取症状均取得了显著改进。对于59条单症状笔记，准确率从0.51提升至0.71，精确率从0.52提升至0.82，召回率从0.52提升至0.72，F1分数从0.49提升至0.73。在375条多症状笔记中，准确率从0.24提升至0.43，精确率从0.6提升至0.76，召回率从0.24提升至0.43，F1分数从0.20提升至0.44。这些结果证明了先进提示工程在LLMs中用于放射肿瘤学领域的有效性。
英文摘要：
This study introduces a novel teacher-student architecture utilizing Large Language Models (LLMs) to improve prostate cancer radiotherapy symptom extraction from clinical notes. Mixtral, the student model, initially extracts symptoms, followed by GPT-4, the teacher model, which refines prompts based on Mixtral's performance. This iterative process involved 294 single symptom clinical notes across 12 symptoms, with up to 16 rounds of refinement per epoch. Results showed significant improvements in extracting symptoms from both single and multi-symptom notes. For 59 single symptom notes, accuracy increased from 0.51 to 0.71, precision from 0.52 to 0.82, recall from 0.52 to 0.72, and F1 score from 0.49 to 0.73. In 375 multi-symptom notes, accuracy rose from 0.24 to 0.43, precision from 0.6 to 0.76, recall from 0.24 to 0.43, and F1 score from 0.20 to 0.44. These results demonstrate the effectiveness of advanced prompt engineering in LLMs for radiation oncology use.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
