arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.08094cs.CYcs.AI
收藏

MedThink: 通过教师引导的推理校正提升小模型的诊断准确性

MedThink: Enhancing Diagnostic Accuracy in Small Models via Teacher-Guided Reasoning Correction

School of Computer Science, Fudan University, Shanghai, China(复旦大学计算机学院,上海,中国)

Xinchun Su, Chunxu Luo, Lipeng Ma, Yixuan Li, Weidong Yang

AI总结:

MedThink通过两阶段知识蒸馏框架提升小语言模型的临床推理能力,在医疗基准测试和胃肠病数据集上均优于传统方法,提升诊断准确率和泛化能力。

摘要:

准确的临床诊断需要广泛的领域知识和复杂的临床推理能力。尽管大语言模型(LLMs)在临床推理方面具有巨大潜力,但其高计算和内存需求限制了在资源受限环境中的部署。知识蒸馏(KD)可以将LLM能力压缩到较小的模型中,但传统KD仅转移表面答案模式,未能保留可靠的诊断所需的结构化推理。为此,我们提出了一个两阶段蒸馏框架MedThink,旨在培养小语言模型(SLMs)的稳健临床推理能力。第一阶段,教师LLM筛选数据并注入领域知识解释以微调学生模型,建立知识基础。第二阶段,教师评估学生的错误,生成连接知识到正确答案的推理链,并通过第二次微调优化学生的诊断推理。我们在通用医疗基准测试和包含955个问题-答案对的胃肠病数据集上评估MedThink。实验表明,MedThink在所有基准测试中均优于六种蒸馏策略:在通用任务中,相对于学生基线提升高达12.7%,在胃肠病评估中达到总准确率56.4%。这表明,以推理为中心的迭代蒸馏可以显著提升SLMs的诊断准确率和泛化能力,同时保持计算效率。我们的代码和数据在https://github.com/destinybird/PrecisionBoost上公开。

英文摘要:

Accurate clinical diagnosis requires extensive domain knowledge and complex clinical reasoning capabilities. Although large language models (LLMs) hold great potential for clinical reasoning, their high computational and memory requirements limit their deployment in resource-constrained environments. Knowledge distillation (KD) can compress LLM capabilities into smaller models, but traditional KD merely transfers superficial answer patterns and fails to preserve the structured reasoning required for reliable diagnosis. To address this, we propose a two-stage distillation framework, MedThink, designed to cultivate robust clinical reasoning in small language models (SLMs). In the first stage, a teacher LLM screens data and injects domain-knowledge explanations to fine-tune a student model, establishing a knowledge foundation. In the second stage, the teacher evaluates the student's errors, generates reasoning chains linking knowledge to correct answers, and refines the student's diagnostic reasoning through a second round of fine-tuning. We evaluate MedThink on general medical benchmarks and a gastroenterology dataset comprising 955 question-answer pairs. Experiments demonstrate that MedThink outperforms six distillation strategies in all benchmarks: achieving an improvement of up to 12.7% over the student baseline in general tasks, and reaching a total top accuracy of 56.4% in gastroenterology evaluation. This indicates that iterative distillation centered on reasoning can significantly enhance the diagnostic accuracy and generalization capabilities of SLMs whilst maintaining computational efficiency. Our code and data are publicly available at https://github.com/destinybird/PrecisionBoost.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑