arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.09213cs.CL
收藏

FineMedLM-o1:从监督微调至测试时训练增强大语言模型的医学知识推理能力

FineMedLM-o1: Enhancing Medical Knowledge Reasoning Ability of LLM from Supervised Fine-Tuning to Test-Time Training

Fudan University(复旦大学);Children’s Hospital of Fudan University(复旦大学附属儿童医院)

Hongzhou Yu, Tianhao Cheng, Yingwen Wang, Wen He, Qing Wang, Ying Cheng, Yuejie Zhang, Rui Feng, Xiaobo Zhang

AI总结:

针对现有医学大模型深度推理能力不足的问题,提出FineMedLM-o1,结合合成数据开展SFT与DPO训练,并首次引入医学领域测试时训练,显著提升医学推理性能,同时推出高质量医学对话合成数据集。

摘要:

大语言模型(LLM)的最新进展已在疾病诊断、治疗方案制定等医学应用中展现出潜力。然而,现有多数医学LLM难以应对复杂医学问题所需的深度推理,例如鉴别诊断和用药推荐。我们提出FineMedLM-o1,利用高质量医学合成数据与长链推理数据开展监督微调(SFT)和直接偏好优化(DPO),使其具备高级对话与深度推理能力。此外,我们首次将测试时训练(TTT)引入医学领域,助力领域适配并确保推理可靠准确。实验结果表明,在核心医学基准上,FineMedLM-o1的平均性能较先前模型提升23%。引入TTT还可带来额外14%的性能提升,凸显其在增强医学推理能力方面的有效性。为支撑这一流程,我们还提出一种新颖的医学对话合成方法。与其他开源数据集相比,我们的数据集在质量和复杂度上均更具优势。项目及数据将在GitHub上发布。

英文摘要:

Recent advancements in large language models (LLMs) have shown promise in medical applications such as disease diagnosis and treatment planning. However, most existing medical LLMs struggle with the deep reasoning required for complex medical problems, such as differential diagnosis and medication recommendations. We propose FineMedLM-o1, which leverages high-quality medical synthetic data and long-form reasoning data for Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO), enabling advanced dialogue and deep reasoning capabilities. Additionally, we introduce Test-Time Training (TTT) in the medical domain for the first time, facilitating domain adaptation and ensuring reliable, accurate reasoning. Experimental results demonstrate that FineMedLM-o1 achieves a 23% average performance improvement over prior models on key medical benchmarks. Furthermore, the introduction of TTT provides an additional 14% performance boost, highlighting its effectiveness in enhancing medical reasoning capabilities. To support this process, we also propose a novel method for synthesizing medical dialogue. Compared to other open-source datasets, our dataset stands out as superior in both quality and complexity. The project and data will be released on GitHub.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑