LingLanMedBench: A Unified Benchmark for Evaluating Large Language Models in Traditional Chinese Medicine
Department of Computer Science and Technology, Beijing Jiaotong University, Beijing, China(北京交通大学计算机科学与技术学院);Tianjin Tasly Digital Chinese Medicine Technology Co., Ltd.(天津塔斯丽数字中医科技有限公司);Tasly Biopharmaceuticals Co., Ltd.(塔斯丽生物医药有限公司);State Key Laboratory of Chinese Medicine Modernization, Tianjin, China(中药现代化国家工程实验室,天津,中国);Institute of Liver Diseases, Hubei Key Laboratory of the theory and application research of liver and kidney in traditional Chinese medicine, Hubei Provincial Hospital of Traditional Chinese Medicine, Wuhan, China(肝病研究所,湖北省中医肝肾理论与应用研究重点实验室,湖北省中医药研究院,武汉,中国);Affiliated Hospital of Hubei University of Chinese Medicine, Wuhan, China(湖北中医药大学附属医院,武汉,中国);Hubei Province Academy of Traditional Chinese Medicine, Wuhan, China(湖北省中医药研究院,武汉,中国);Department of Gastroenterology, Guang’anmen Hospital, China Academy of Chinese Medical Sciences, Beijing, China(消化内科,广安门医院,中国中医科学院,北京,中国);China Academy of Chinese Medical Sciences, Beijing, China(中国中医科学院,北京,中国);China Institute for History of Medicine and Medical Literature, China Academy of Chinese Medical Sciences, Beijing, China(中国中医科学院中国医学史与医学文献研究所,北京,中国);Beijing Research Institute of Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China(北京中医研究院,北京中医药大学,北京,中国);Beijing University of Chinese Medicine Third Affiliated Hospital, Beijing University of Chinese Medicine, Beijing 100029, China(北京中医药大学第三附属医院,北京中医药大学,北京100029,中国)
Reliable evaluation of large language models in traditional Chinese medicine remains limited because existing benchmarks mainly focus on examination-style question answering or isolated generative tasks. We developed LingLanMedBench, a unified benchmark comprising 25,620 expert-verified items across 13 subtasks and five domains, covering knowledge, language understanding, information extraction, and diagnostic-therapeutic decision-making. LingLanMedBench standardizes task formats and evaluation metrics and further incorporates synonym-tolerant clinical-label matching, empirically derived hard subsets, and decision-recognition reformulations of open-ended clinical tasks. Fourteen open-source and proprietary models were evaluated under a standardized zero-shot protocol. Performance varied substantially across domains, model families, task formulations, and difficulty levels. The leading overall score decreased from 51.1 on the full benchmark to 31.9 on the hard subsets. Prescription performance also differed markedly between open-ended generation, with F1 scores of 32.7 and 20.3, and decision recognition, with accuracies of 90.6 and 56.8 on the full and hard subsets, respectively. These findings reveal a substantial gap between knowledge-oriented performance and clinically faithful reasoning. LingLanMedBench provides a standardized and reproducible framework for characterizing current capabilities, identifying systematic limitations, and tracking meaningful progress in large language models for traditional Chinese medicine.
LingLan: An Advancing Traditional Chinese Medicine Diagnosis LLM with Multimodal Data
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Guangdong Provincial Laboratory of Traditional Chinese Medicine Hengqin(广东省中医药横琴实验室)
Enhancing the Traditional Chinese Medicine Capabilities of Large Language Model through Reinforcement Learning from AI Feedback
School of Computer and Information Science, Southwest University(西南大学计算机与信息科学学院);School of Computing Technologies, RMIT University(RMIT大学计算技术学院);Yongdingmenwai Community Health Service Center(永定门外社区卫生服务中心)
Qibo: A Large Language Model for Traditional Chinese Medicine
College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);Evidence-Based Medicine Center, Tianjin University of Traditional Chinese Medicine(天津中医药大学循证医学中心);First Teaching Hospital of Tianjin University of Traditional Chinese Medicine(天津中医药大学第一附属医院);National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion(国家中医针灸临床医学研究中心);Tiandazhitu(天大地图)
ZhiFangDanTai: Fine-tuning Graph-based Retrieval-Augmented Generation Model for Traditional Chinese Medicine Formula
School of Management, Capital Normal University(管理学院,首都师范大学);School of Electrical Engineering and Computer Science, the University of Queensland(电气工程与计算机科学学院,昆士兰大学)
Tianyi: A Traditional Chinese Medicine all-rounder language model and its Real-World Clinical Practice
Nanjing University of Chinese Medicine(南京中医药大学);Beijing University of Chinese Medicine(北京中医药大学);Capital Medical University(首都医科大学);Zhejiang Lab(之江实验室);China Academy of Chinese Medical Sciences(中国中医科学院)
MTCMB: A Multi-Task Benchmark Framework for Evaluating LLMs on Knowledge, Reasoning, and Safety in Traditional Chinese Medicine
School of Software Engineering, Sun Yat-sen University(中山大学软件学院);Institute of TCM Diagnostics, Hunan University of Chinese Medicine(湖南中医药大学中医诊断研究所);School of Artificial Intelligence, Guilin University of Electronic Technology(桂林电子科技大学人工智能学院);Department of Computer Science, Cornell University(康奈尔大学计算机科学系)
Evidence-Based Intelligent Diagnostic and Therapeutic Visualization System with Large Language Models: Multi-Turn Interaction and Multimodal Treatment Plan Generation
Harbin Institute of Technology, Weihai(哈尔滨工业大学(威海));Harbin Institute of Technology (Weihai) Qingdao Research Institute(哈尔滨工业大学(威海)青岛研究院);Shandong Key Laboratory of Digital Service Computing Technology and Systems(山东省数字服务计算技术与系统重点实验室);Weihai Municipal Hospital(威海市人民医院);Shanghai Taizhu Technology Co., Ltd(上海泰山技术有限公司);Tianjin Zhifu Qihuang Medical Technology Co., Ltd(天津中孚启黄医疗技术有限公司)
TianHui: A Domain-Specific Large Language Model for Diverse Traditional Chinese Medicine Scenarios
School of Intelligent Medicine, Chengdu University of Traditional Chinese Medicine(成都中医药大学智能医学学院);The Acupuncture and Tuina School, Chengdu University of Traditional Chinese Medicine(成都中医药大学针灸推拿学院);Center of Preventive Medicine, Hospital of Chengdu University of Traditional Chinese Medicine(成都中医药大学附属医院预防医学中心)
Leveraging Group Relative Policy Optimization to Advance Large Language Models in Traditional Chinese Medicine
University of Missouri(密苏里大学);Christopher S. Bond Life Sciences Center, University of Missouri(密苏里大学克里斯托弗·S·邦德生命科学中心);Community Health Service Center Shanghai Pudong New Area(上海市浦东新区社区卫生服务中心);Shanghai University of Traditional Chinese Medicine(上海中医药大学)
BianCang: A Traditional Chinese Medicine Large Language Model
Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), and with Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science(计算机功率网络与信息安全重点实验室,教育部,山东计算机科学中心(国家超级计算机中心济南中心),齐鲁工业大学(山东科学院),并与山东省计算功率互联网与服务计算省级重点实验室,山东省计算机科学基础研究中心);Australian Artificial Intelligence Institute, University of Technology Sydney(澳大利亚人工智能研究所,技术悉尼大学);Affiliated Hospital of Shandong University of Traditional Chinese Medicine(山东中医药大学附属医院);College of Information Science and Technology & Artificial Intelligence, Nanjing Forestry University(信息科学与技术及人工智能学院,南京林业大学);Department of ICT and Center for AI Research, University of Agder (UiA)(ICT系和人工智能研究中心,阿格德大学(UiA))
TCMBench: A Comprehensive Benchmark for Evaluating Large Language Models in Traditional Chinese Medicine
Shanghai Institute of AI for Education, East China Normal University(华东师范大学人工智能教育研究院);School of Pharmacy, East China Normal University(华东师范大学药学院);Shanghai Skin Disease Hospital, School of Medicine, Tongji University(同济大学医学院附属上海市皮肤病医院)
ShizhenGPT: Towards Multimodal LLMs for Traditional Chinese Medicine
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Freedom AI;National Health Data Institute, Shenzhen(深圳国家健康数据研究院);Shenzhen Research Institute of Big Data(深圳大数据研究院);Guangdong University of Technology(广东工业大学)
Lingdan-V2: Large language models for clinically grounded reasoning in traditional Chinese medicine
School of Computer Science & Technology, Beijing Jiaotong University(北京交通大学计算机科学与技术学院);Tianjin Tasly Digital Chinese Medicine Technology Co., Ltd.(天津天士力数字中医药科技有限公司);Tasly Biopharmaceuticals Co., Ltd.(天士力生物制药有限公司);Affiliated Hospital of Hubei University of Chinese Medicine(湖北中医药大学附属医院);Da’an Health Technology (Beijing) Co., Ltd.(大安健康科技(北京)有限公司);Institute of Chinese Medicine Epidemic Disease, Beijing University of Chinese Medicine(北京中医药大学中医疫病研究所);Guang’anmen Hospital, China Academy of Chinese Medical Sciences(中国中医科学院广安门医院);Xiyuan Hospital, China Academy of Chinese Medical Sciences(中国中医科学院西苑医院)
BenCao: An Instruction-Tuned Large Language Model for Traditional Chinese Medicine
Community Health Service Center Shanghai Pudong New Area(上海浦东新区社区卫生服务中心);School of Acupuncture-Moxibustion and Tuina, Shanghai University of Traditional Chinese Medicine(上海中医药大学针灸推拿学院)
本文提出基于 ChatGPT 的中医药多模态助手 BenCao,通过自然语言指令微调、知识库、思维链模拟、专家反馈及外部 API 整合,提升诊断、药材识别与体质分类性能并实现部署。
An Ontology-Based Artificial Intelligence Model for Medicine Side-Effect Prediction: Taking Traditional Chinese Medicine as An Example
School of Information and Software Engineering, University of Electronic Science and Technology of China(电子科技大学信息与软件工程学院);School of Electrical Engineering and Telecommunications, University of New South Wales(新南威尔士大学电气工程与电信学院);Faculty of Medicine, Ludwig Maximilian University of Munich(慕尼黑大学医学院);The First Clinical Medical College, Zhejiang Chinese Medicine University(浙江中医药大学第一临床医学院)
MedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models
East China Normal University(华东师范大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Hasso Plattner Institute(哈索·普拉特纳研究所);University of Potsdam(波茨坦大学);Cooperative Medianet Innovation Center, Shanghai Jiao Tong University(上海交通大学合作媒体网络创新中心)
CMMMU: A Chinese Massive Multi-discipline Multimodal Understanding Benchmark
Multimodal Art Projection Research Community(多模态艺术投影研究共同体);Hong Kong University of Science and Technology(香港科技大学);University of Waterloo(滑铁卢大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Peking University(北京大学);The Hong Kong Polytechnic University(香港理工大学);Waseda University(早稻田大学);University of Manchester(曼彻斯特大学)
TCM-DiffRAG: Personalized Syndrome Differentiation Reasoning Method for Traditional Chinese Medicine based on Knowledge Graph and Chain of Thought
Macao Polytechnic University(澳门理工大学);Guangdong Institute of Intelligence Science and Technology(广东智能科学技术研究院);Zhejiang Chinese Medical University(浙江中医药大学);Hangzhou Ganzhicao Technology Co., Ltd(杭州甘之草科技有限公司);BoardWare Information System Limited(BoardWare信息系统有限公司)
An Interpretable AI framework Quantifying Traditional Chinese Medicine Principles Towards Enhancing and Integrating with Modern Biomedicine
Hong Kong Baptist University(香港浸会大学);Nanyang Technological University(南洋理工大学);The Chinese Medicine Hospital of Hong Kong(香港中医医院);Westlake University(西湖大学)
Jingfang: An LLM-Based Multi-Agent System for Precise Medical Consultation and Syndrome Differentiation in Traditional Chinese Medicine
Beijing University of Posts and Telecommunications(北京邮电大学);Southeast University(东南大学);University of Chinese Academy of Sciences(中国科学院大学);University of Science and Technology of China(中国科学技术大学)
LingShu: A Large-Scale Symptom-Centric Contextualized Knowledge Graph Bridging Traditional Chinese Medicine and Modern Biomedicine
Beijing Jiaotong University(北京交通大学);Hubei Provincial Hospital of Traditional Chinese Medicine(湖北省中医院);Hubei University of Chinese Medicine(湖北中医药大学);Hubei Province Academy of Traditional Chinese Medicine(湖北省中医药研究院);China Academy of Chinese Medical Sciences(中国中医科学院);Xiyuan Hospital(西苑医院);National Clinical Research Center for Chinese Medicine Cardiology(国家中医心血管病临床医学研究中心);Hubei Provincial Clinical Research Center for Acupuncture and Moxibustion in Obesity Treatment(湖北省肥胖病针灸临床研究中心);Hubei Shizhen Laboratory(湖北时珍实验室);Tianjin Ta(天津(此处原文未完整,按原文提取))
TCMClinicalReason-Bench: Can Language Models Reason from Pathogenesis to Prescription over Real-World Clinical Cases?
Nanjing University of Chinese Medicine(南京中医药大学);Beijing University of Chinese Medicine(北京中医药大学);Johns Hopkins University(约翰斯·霍普金斯大学);Huzhou University(湖州大学)
From Physician Expertise to Clinical Agents: Preserving, Standardizing, and Scaling Physicians' Medical Expertise with Lightweight LLM
the School of Chinese Medicine, the Beijing University of Chinese Medicine, Beijing, China(中国中医药大学中医学院,北京中医药大学,北京,中国);the School of Pharmacy, Nanjing University of Chinese Medicine, Nanjing, China(南京中医药大学药学院,南京,中国);the Infectious disease department, Dongfang Hospital, Beijing University of Chinese Medicine, Beijing, China(北京中医药大学东四医院感染科,北京,中国);the Gulou Hospital of Traditional Chinese Medicine of Beijing,Beijing, China(北京中医药大学附属鼓楼中医医院,北京,中国);the Department of Computer Science, Johns Hopkins University, Baltimore, USA(约翰霍普金斯大学计算机科学系,美国马里兰州巴尔的摩市);the Department of Education, Dongzhimen Hospital, Beijing University of Chinese Medicine, Beijing, China(北京中医药大学东直门医院教育部,北京,中国);the Department of Pediatrics, Wangjing Hospital, China Academy of Chinese Medical Sciences, Beijing, China(中国医学科学院北京协和医院儿科部,北京,中国);the School of Information Engineering, Huzhou University, Huzhou, China(湖州大学信息工程学院,湖州,中国);Research Center for Scientific Data Hub, Zhejiang Lab, Hangzhou, China(浙江省实验室科学数据中心研究中心,杭州,中国);the Frontier Basic Research Center, Zhejiang Lab, Hangzhou, China(浙江省实验室前沿基础研究中心,杭州,中国);the Research Center for High Efficiency Computing Infrastructure, Zhejiang Lab, Hangzhou, China(浙江省实验室高效计算基础设施研究中心,杭州,中国)
TCM-FTP: Fine-Tuning Large Language Models for Herbal Prescription Prediction
The Hong Kong University of Science and Technology(香港科技大学);Beijing Jiaotong University(北京交通大学);NVIDIA(英伟达);Henan University of Chinese Medicine(河南中医药大学)
Performance Evaluation of Lightweight Open-source Large Language Models in Pediatric Consultations: A Comparative Analysis
Children’s Hospital of Chongqing Medical University(重庆医科大学附属儿童医院);Stanford University School of Medicine(斯坦福大学医学院);Chongqing University of Technology(重庆理工大学);Columbia University(哥伦比亚大学)
FEAT: A Multi-Agent Forensic AI System with Domain-Adapted Large Language Model for Automated Cause-of-Death Analysis
Key Laboratory of National Ministry of Health for Forensic Sciences, School of Medicine \& Forensics, Health Science Center, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China;School of Mathematics;Statistics, Xi'an Jiaotong University, Xi'an, Shaanxi 710149, China;Hebei Key Laboratory of Forensic Medicine, College of Forensic Medicine, Hebei Medical University, Shijiazhuang, Hebei 050017, China;Key Laboratory of Biomedical Information Engineering of Ministry of Education, School of Life Science;Technology, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China;Faculty of Forensic Medicine, Guangdong Province Translational Forensic Medicine Engineering Technology Research Center, Zhongshan School of Medicine, Sun Yat-Sen University, Guangzhou, Guangdong 510080, China;Department of Forensic Medicine, Xinxiang Medical University, Xinxiang, Henan 453003, China;Department of Forensic Medicine, School of Basic Medical Sciences, Fudan University, Shanghai 200032, China;Department of Forensic Medicine, Tongji Medical College, Huazhong University of Science;Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai 200063, China;School of Forensic Medicine of Jining Medical University, Jining, Shandong 272067, China