Recently, large language model (LLM) based artificial intelligence (AI) systems have demonstrated remarkable capabilities in natural language understanding and generation. However, these models face a significant challenge when it comes to sensitive applications, such as reasoning over medical knowledge and answering medical questions in a physician-like manner. Prior studies attempted to overcome this challenge by increasing the model size (>100B) to learn more general medical knowledge, while there is still room for improvement in LLMs with smaller-scale model sizes (<100B). In this work, we start from a pre-trained general LLM model (AntGLM-10B) and fine-tune it from a medical beginner towards a medical expert (called AntGLM-Med-10B), which leverages a 3-stage optimization procedure, i.e., general medical knowledge injection, medical domain instruction tuning, and specific medical task adaptation. Our contributions are threefold: (1) We specifically investigate how to adapt a pre-trained general LLM in medical domain, especially for a specific medical task. (2) We collect and construct large-scale medical datasets for each stage of the optimization process. These datasets encompass various data types and tasks, such as question-answering, medical reasoning, multi-choice questions, and medical conversations. (3) Specifically for multi-choice questions in the medical domain, we propose a novel Verification-of-Choice approach for prompting engineering, which significantly enhances the reasoning ability of LLMs. Remarkably, by combining the above approaches, our AntGLM-Med-10B model can outperform the most of LLMs on PubMedQA, including both general and medical LLMs, even when these LLMs have larger model size.
Automating Expert-Level Medical Reasoning Evaluation of Large Language Models
University of Minnesota(明尼苏达大学);University of Georgia(佐治亚大学);University of Virginia(弗吉尼亚大学);Columbia University(哥伦比亚大学);University of California San Francisco(加利福尼亚大学旧金山分校);University of Colorado Anschutz Medical Campus(科罗拉多大学安舒茨医学中心)
A foundation model for human-AI collaboration in medical literature mining
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);National Library of Medicine, National Institutes of Health(美国国立卫生研究院国家医学图书馆);National Institute of Diabetes and Digestive and Kidney Diseases, National Institutes of Health(美国国立卫生研究院国家糖尿病、消化和肾脏疾病研究所);Massachusetts General Hospital(麻省总医院);Harvard Medical School(哈佛医学院);National Eye Institute, National Institutes of Health(美国国立卫生研究院国家眼科研究所);Northwestern University Feinberg School of Medicine(西北大学费恩伯格医学院);Chungbuk National University Hospital(忠北国立大学医院);Chungbuk National University College of Medicine(忠北国立大学医学院);University of Pittsburgh Medical Center(匹兹堡大学医学中心);Weill Cornell Medicine(威尔康奈尔医学院);UT Southwestern Medical Center(德克萨斯大学西南医学中心);University of Washington(华盛顿大学);NYU Langone Health(纽约大学朗格尼健康中心);OSF HealthCare Illinois Neurological Institute(OSF医疗保健伊利诺伊神经研究所)
Structured Outputs Enable General-Purpose LLMs to be Medical Experts
University of Bristol(布里斯托大学);Lehigh University(利哈伊大学);The University of Queensland(昆士兰大学);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);University of California, Merced(加利福尼亚大学默塞德分校)
SparseDoctor: Towards Efficient Chat Doctor with Mixture of Experts Enhanced Large Language Models
Department of Computer Science, Hong Kong Chu Hai College(香港 Chu Hai 学院计算机科学系);Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算机系)
AIRI;Moscow Institute of Physics and Technology(莫斯科物理技术学院);Skokovo Institute of Science and Technology(斯科克沃科学与技术研究所);Innopolis University(因诺普里斯大学);SberHealth;Lomonosov Moscow State University(罗蒙诺索夫莫斯科国立大学)
LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validation
College of Computer Science and Artificial Intelligence, Fudan University(计算机科学与人工智能学院,复旦大学);Shanghai Key Lab of Intelligent Information Processing(上海智能信息处理重点实验室);Shanghai AI Laboratory(上海人工智能实验室);Northwestern University(西北大学)
ECG-Expert-QA: A Benchmark for Evaluating Medical Large Language Models in Heart Disease Diagnosis
Zhejiang University School of Medicine(浙江大学医学院);The First Affiliated Hospital(第一附属医院);Liangzhu Laboratory(良渚实验室);Shandong Jianzhu University(山东建筑大学);FUXI AI Lab(伏羲人工智能实验室);Beijing Normal University(北京师范大学);Southern University of Science and Technology(南方科技大学);China University of Geosciences (Wuhan)(中国地质大学(武汉));Hangzhou Baorun Biotechnology Co., Ltd.(杭州宝润生物科技有限公司)
Emulating Human Cognitive Processes for Expert-Level Medical Question-Answering with Large Language Models
Dell Medical School at the University of Texas(德克萨斯大学戴尔医学院);William Beaumont University Hospital(威廉·博蒙特大学医院);Austin State Hospital(奥斯汀州立医院);Institute of Mental Health, University of Nottingham(诺丁汉大学精神卫生研究所);Government Medical College and Hospital, Chandigarh(昌迪加尔政府医学院及医院);University of Augsburg(奥格斯堡大学);McGovern Medical School and The University of Texas Health Science Center(麦戈文医学院与德克萨斯大学健康科学中心);Sheikh Shakhbout Medical City(谢赫·沙赫布特医疗城);Dr Akbar Niazi Teaching Hospital(阿克巴尔·尼亚兹博士教学医院);Centro Médico ABC(ABC医疗中心);Houston Regional Gastroenterology Institute(休斯顿地区胃肠病学研究所);NeuroReef Labs(NeuroReef实验室)
PrinciplismQA: A Philosophy-Grounded Approach to Assessing LLM-Human Clinical Medical Ethics Alignment
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));National Health Data Institute, Shenzhen(深圳国家健康数据研究院);Shenzhen Research Institute of Big Data(深圳大数据研究院)
MedLayBench-V: A Large-Scale Benchmark for Expert-Lay Semantic Alignment in Medical Vision Language Models
Seoul National University(首尔国立大学);Seoul National University College of Medicine(首尔国立大学医学院);Department of Radiology, Seoul National University Hospital(首尔国立大学医院放射科);Healthcare AI Research Institute, Seoul National University Hospital(首尔国立大学医院健康人工智能研究所);The Advanced Imaging and Computational Neuroimaging (AICON) Laboratory(先进影像与计算神经影像实验室)
SOLVE-Med: Specialized Orchestration for Leading Vertical Experts across Medical Specialties
University of Naples Federico II, DIETI, Naples, Italy(那不勒斯费德里科二世大学,DIETI,那不勒斯,意大利);Northwestern University, Department of Computer Science, Evanston, IL, United States(西北大学,计算机科学系,伊利诺伊州埃文斯顿,美国)
MEDLAYXPLAIN: Benchmarking the Expert-Lay Gap in Medical Vision-Language Models
Seoul National University(首尔大学);Seoul National University Hospital(首尔大学医院);Seoul National University College of Medicine(首尔大学医学院);Sungkyunkwan University School of Medicine(成均馆大学医学院)
When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications
City University of Hong Kong(香港城市大学);Jarvis Research Center(贾维斯研究中心);Tencent YouTu Lab(腾讯优图实验室);Southern University of Science and Technology(南方科技大学);University of Science and Technology of China(中国科学技术大学)