用于癌症毒性症状提取的混合师生大型语言模型精炼
Hybrid Student-Teacher Large Language Model Refinement for Cancer Toxicity Symptom Extraction
AI总结：
本文提出混合师生架构，由 GPT-4o 指导 Zephyr-7b-beta 与 Phi3-mini-128 动态选择提示词精炼、RAG或微调，以低成本显著提升癌症放疗毒性症状提取性能。
摘要：
大型语言模型（LLM）在临床症状提取方面具有巨大潜力，但其在医疗环境中的部署受到隐私问题、计算限制和运营成本的制约。本研究采用一种新颖的迭代精炼方法，研究用于癌症毒性症状提取的紧凑型LLM优化。我们采用师生架构，使用 Zephyr-7b-beta 和 Phi3-mini-128 作为学生模型，GPT-4o 作为教师模型，在提示词精炼、检索增强生成（RAG）和微调策略之间进行动态选择。我们在覆盖12种放疗后毒性症状的294份临床笔记上进行的实验证明了该方法的有效性。RAG方法被证明效率最高，在精炼过程中将 Zephyr-7b-beta 的平均准确率从0.32提升至0.73，将 Phi3-mini-128 的平均准确率从0.40提升至0.87。在测试集中，两个模型在各症状上的准确率均提高约0.20。值得注意的是，实现这一提升的成本：Zephyr 比 GPT-4o 低45倍，Phi-3 比 GPT-4o 低79倍。这些结果凸显了迭代精炼技术在增强紧凑型LLM临床应用能力方面的潜力，可在医疗环境中兼顾性能、成本效益和隐私保护。
英文摘要：
Large Language Models (LLMs) offer significant potential for clinical symptom extraction, but their deployment in healthcare settings is constrained by privacy concerns, computational limitations, and operational costs. This study investigates the optimization of compact LLMs for cancer toxicity symptom extraction using a novel iterative refinement approach. We employ a student-teacher architecture, utilizing Zephyr-7b-beta and Phi3-mini-128 as student models and GPT-4o as the teacher, to dynamically select between prompt refinement, Retrieval-Augmented Generation (RAG), and fine-tuning strategies. Our experiments on 294 clinical notes covering 12 post-radiotherapy toxicity symptoms demonstrate the effectiveness of this approach. The RAG method proved most efficient, improving average accuracy scores from 0.32 to 0.73 for Zephyr-7b-beta and from 0.40 to 0.87 for Phi3-mini-128 during refinement. In the test set, both models showed an approximate 0.20 increase in accuracy across symptoms. Notably, this improvement was achieved at a cost 45 times lower than GPT-4o for Zephyr and 79 times lower for Phi-3. These results highlight the potential of iterative refinement techniques in enhancing the capabilities of compact LLMs for clinical applications, offering a balance between performance, cost-effectiveness, and privacy preservation in healthcare settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
