Accurate clinical diagnosis requires extensive domain knowledge and complex clinical reasoning capabilities. Although large language models (LLMs) hold great potential for clinical reasoning, their high computational and memory requirements limit their deployment in resource-constrained environments. Knowledge distillation (KD) can compress LLM capabilities into smaller models, but traditional KD merely transfers superficial answer patterns and fails to preserve the structured reasoning required for reliable diagnosis. To address this, we propose a two-stage distillation framework, MedThink, designed to cultivate robust clinical reasoning in small language models (SLMs). In the first stage, a teacher LLM screens data and injects domain-knowledge explanations to fine-tune a student model, establishing a knowledge foundation. In the second stage, the teacher evaluates the student's errors, generates reasoning chains linking knowledge to correct answers, and refines the student's diagnostic reasoning through a second round of fine-tuning. We evaluate MedThink on general medical benchmarks and a gastroenterology dataset comprising 955 question-answer pairs. Experiments demonstrate that MedThink outperforms six distillation strategies in all benchmarks: achieving an improvement of up to 12.7% over the student baseline in general tasks, and reaching a total top accuracy of 56.4% in gastroenterology evaluation. This indicates that iterative distillation centered on reasoning can significantly enhance the diagnostic accuracy and generalization capabilities of SLMs whilst maintaining computational efficiency. Our code and data are publicly available at https://github.com/destinybird/PrecisionBoost.
CAST: Non-Privileged Clipped Asymmetric Self-Teaching with Advantage Flipping for GRPO
School of Software and Microelectronics, Peking University, Beijing(北京大学软件与微电子学院);School of Artificial Intelligence, Peking University, Beijing(北京大学人工智能学院);School of Computer Science, Peking University, Beijing(北京大学计算机科学学院);School of Future Technology, Peking University, Beijing(北京大学未来技术学院)
Language Models as Semantic Teachers: Post-Training Alignment for Medical Audio Understanding
Eindhoven University of Technology, The Netherlands(埃因霍温理工大学,荷兰);Kyutai, France(Kyutai公司,法国);Eindhoven Artificial Intelligence Systems Institute, The Netherlands(埃因霍温人工智能系统研究所,荷兰)
DrawSim-PD: Simulating Student Science Drawings to Support NGSS-Aligned Teacher Diagnostic Reasoning
Department of Computer Science, Drexel University(德雷塞尔大学计算机科学系);College of Chemistry, Beijing Normal University(北京师范大学化学学院);Department of Teaching and Learning, Washington State University(华盛顿州立大学教学与学习系)
Explainable Collaborative Problem Solving Diagnosis with BERT using SHAP and its Implications for Teacher Adoption
UCL Knowledge Lab, Institute of Education, University College London, UK(伦敦大学学院教育学院知识实验室);UCL Centre for Artificial Intelligence, Department of Computer Science, University College London, UK(伦敦大学学院人工智能中心)
From Atoms to Chains: Divergence-Guided Reasoning Curriculum for Unlabeled LLM Domain Adaptation
Beijing Key Laboratory of Intelligent Information Technology, School of Computer Science and Technology, Beijing Institute of Technology(北京智能信息科技重点实验室,计算机科学与技术学院,北京理工大学);ByteDance China(字节跳动中国)
Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Work
School of Computing, KAIST, Daejeon, Republic of Korea(韩国庆熙大学计算机学院);University of Michigan, Ann Arbor, USA(美国密歇根大学);Ewha Womans University, Seoul, Republic of Korea(韩国成均馆大学);AlgorithmLabs, Seoul, Republic of Korea(韩国AlgorithmLabs公司)
Beijing University of Posts and Telecommunications(北京邮电大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Southwestern University of Finance and Economics(西南财经大学)
Knowledge and Reasoning Augmented Learning for Language Model Assisted Antimicrobial Decision Support
Information Center, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital(信息中心、复杂严重罕见疾病国家重点实验室、北京友谊医院);Department of Critical Care Medicine, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Peking Union Medical College and Chinese Academy of Medical Sciences(重症医学科、复杂严重罕见疾病国家重点实验室、北京友谊医院、北京友谊医院和中国医学科学院);Medical Intensive Care Unit, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Peking Union Medical College and Chinese Academy of Medical Sciences(医疗重症病房、复杂严重罕见疾病国家重点实验室、北京友谊医院、北京友谊医院和中国医学科学院)
The Correct Answer Trap: Pedagogically-Grounded Detection and Feedback for Hidden Misconceptions
Department of Computer Science, University College London, The United Kingdom(伦敦大学学院计算机科学系,英国);Centre for Artificial Intelligence, University College London, The United Kingdom(伦敦大学学院人工智能中心,英国)