arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.16563cs.AI
收藏

通过在弱标签上微调轻量级LLM增强放射学报告中的疾病检测

Enhancing disease detection in radiology reports through fine-tuning lightweight LLM on weak labels

Weill Cornell Medicine(威尔康奈尔医学院);Thomas Jefferson University Hospital(托马斯杰斐逊大学医院)

Yishu Wei, Xindi Wang, Hanley Ong, Yiliang Zhou, Adam Flanders, George Shih, Yifan Peng

AI总结:

本研究通过使用合成标签微调Llama 3.1-8B等轻量级LLM进行联合任务训练,发现其在开放式疾病检测任务中表现优异,并能超越低质量教师标签,为医疗领域LLM专门化提供了新方向。

摘要:

尽管在将大型语言模型(LLMs)应用于医疗领域方面取得了显著进展,但若干限制仍阻碍其实际应用。其中包括模型大小的限制以及缺乏特定队列的标注数据集。在本研究中,我们探讨了通过使用合成标签的数据集进行微调,以改进轻量级LLM(如Llama 3.1-8B)的潜力。通过合并各自的指令数据集,对两项任务进行了联合训练。当特定任务合成标签的质量相对较高时(例如由GPT-4o生成),Llama 3.1-8B在开放式疾病检测任务上取得了令人满意的性能,micro F1分数为0.91。相反,当任务相关合成标签的质量相对较低时(例如来自MIMIC-CXR数据集),经过微调的Llama 3.1-8B在校准至精选标签后,能够超越其嘈杂的教师标签(micro F1分数为0.67对比0.63),表明了模型强大的内在底层能力。这些发现证明了使用合成标签微调LLMs的潜力,为未来医疗领域LLM专门化研究提供了有前景的方向。

英文摘要:

Despite significant progress in applying large language models (LLMs) to the medical domain, several limitations still prevent them from practical applications. Among these are the constraints on model size and the lack of cohort-specific labeled datasets. In this work, we investigated the potential of improving a lightweight LLM, such as Llama 3.1-8B, through fine-tuning with datasets using synthetic labels. Two tasks are jointly trained by combining their respective instruction datasets. When the quality of the task-specific synthetic labels is relatively high (e.g., generated by GPT4- o), Llama 3.1-8B achieves satisfactory performance on the open-ended disease detection task, with a micro F1 score of 0.91. Conversely, when the quality of the task-relevant synthetic labels is relatively low (e.g., from the MIMIC-CXR dataset), fine-tuned Llama 3.1-8B is able to surpass its noisy teacher labels (micro F1 score of 0.67 v.s. 0.63) when calibrated against curated labels, indicating the strong inherent underlying capability of the model. These findings demonstrate the potential of fine-tuning LLMs with synthetic labels, offering a promising direction for future research on LLM specialization in the medical domain.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑