arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.13893cs.CL
收藏

基于大型语言模型临床概念嵌入的迁移学习

Transfer Learning with Clinical Concept Embeddings from Large Language Models

University of Pittsburgh(匹兹堡大学);GE Healthcare(GE医疗);University of Utah(犹他大学)

Yuhe Gao, Runxue Bao, Yuelyu Ji, Yiming Sun, Chenxi Song, Jeffrey P. Ferraro, Ye Ye

AI总结:

本研究评估了大型语言模型语义嵌入在医疗跨站点迁移学习中的作用,发现特定领域LLMs在本地和直接迁移中表现更优,通用模型需微调,过度调优生物医学嵌入则可能降低效果。

摘要:

知识共享在医疗保健中至关重要,特别是利用多个临床站点的数据来解决数据稀缺、降低成本并实现及时干预。迁移学习可以促进跨站点知识转移,但主要挑战是不同站点临床概念的异构性。大型语言模型(LLMs)在捕捉临床概念的语义并减少异构性方面显示出巨大潜力。本研究分析了来自两个大型医疗保健系统的电子健康记录,以评估LLMs的语义嵌入对本地、共享和迁移学习模型的影响。结果表明,特定领域的LLMs(如Med-BERT)在本地和直接迁移场景中始终表现更好,而像OpenAI嵌入这样的通用模型则需要微调才能达到最佳性能。然而,对带有生物医学嵌入的模型进行过度调优可能会降低有效性,强调需要保持平衡。本研究突出了特定领域嵌入和仔细模型调优对于医疗保健中有效知识转移的重要性。

英文摘要:

Knowledge sharing is crucial in healthcare, especially when leveraging data from multiple clinical sites to address data scarcity, reduce costs, and enable timely interventions. Transfer learning can facilitate cross-site knowledge transfer, but a major challenge is heterogeneity in clinical concepts across different sites. Large Language Models (LLMs) show significant potential of capturing the semantic meaning of clinical concepts and reducing heterogeneity. This study analyzed electronic health records from two large healthcare systems to assess the impact of semantic embeddings from LLMs on local, shared, and transfer learning models. Results indicate that domain-specific LLMs, such as Med-BERT, consistently outperform in local and direct transfer scenarios, while generic models like OpenAI embeddings require fine-tuning for optimal performance. However, excessive tuning of models with biomedical embeddings may reduce effectiveness, emphasizing the need for balance. This study highlights the importance of domain-specific embeddings and careful model tuning for effective knowledge transfer in healthcare.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑