Med42-v2 introduces a suite of clinical large language models (LLMs) designed to address the limitations of generic models in healthcare settings. These models are built on Llama3 architecture and fine-tuned using specialized clinical data. They underwent multi-stage preference alignment to effectively respond to natural prompts. While generic models are often preference-aligned to avoid answering clinical queries as a precaution, Med42-v2 is specifically trained to overcome this limitation, enabling its use in clinical settings. Med42-v2 models demonstrate superior performance compared to the original Llama3 models in both 8B and 70B parameter configurations and GPT-4 across various medical benchmarks. These LLMs are developed to understand clinical queries, perform reasoning tasks, and provide valuable assistance in clinical environments. The models are now publicly available at \href{https://huggingface.co/m42-health}{https://huggingface.co/m42-health}.
University of Wisconsin（威斯康星大学）；Loyola University Chicago（芝加哥洛约拉大学）；Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）
Applying methods in natural language processing on electronic health records (EHR) data is a growing field. Existing corpus and annotation focus on modeling textual features and relation prediction. However, there is a paucity of annotated corpus built to model clinical diagnostic thinking, a process involving text understanding, domain knowledge abstraction and reasoning. This work introduces a hierarchical annotation schema with three stages to address clinical text understanding, clinical reasoning, and summarization. We created an annotated corpus based on an extensive collection of public…
Beyond MedQA: Towards Real-world Clinical Decision Making in the Era of LLMs
Department of Computer Science, Emory University（埃默里大学计算机科学系）；Children’s Healthcare of Atlanta（亚特兰大儿童医院）；Nell Hodgson Woodruff School of Nursing, Emory University（埃默里大学奈尔·霍德森·伍德鲁夫护理学院）
Guiding Clinical Reasoning with Large Language Models via Knowledge Seeds
Zhejiang University（浙江大学）；Tencent Youtu Lab（腾讯优图实验室）；Jarvis Research Center（贾维斯研究中心）；Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）
KREL: Automatic Medical Coding via Knowledge-Guided Reasoning over Clinical Evidence with LLMs
The University of New South Wales（新南威尔士大学）；University of Göttingen（哥廷根大学）；Macquarie University（麦考瑞大学）；Beijing Intelligent Decision Medical Technology Co. Ltd（北京智决医疗科技有限公司）
Enhancing Large Language Models for Clinical Decision Support by Incorporating Clinical Practice Guidelines
University of Pittsburgh（匹兹堡大学）；University of Pittsburgh Medical Center（匹兹堡大学医学中心）；University of Pittsburgh Medical Center Western Maryland（匹兹堡大学医学中心西马里兰分院）
A Decade-Scale Benchmark Evaluating LLMs' Clinical Practice Guidelines Detection and Adherence in Multi-turn Conversations
Microsoft Research Asia（微软亚洲研究院）；Hong Kong University of Science and Technology（香港科技大学）；Peking University（北京大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校）；Emory University（埃默里大学）；University of Minnesota（明尼苏达大学）；University of Massachusetts, Lowell（马萨诸塞大学洛厄尔分校）；UMass Chan Medical School（UMass Chan医学学院）
Systematic Evaluation of the Quality of Synthetic Clinical Notes Rephrased by LLMs at Million-Note Scale
Australian e-Health Research Centre, CSIRO, Australia（澳大利亚电子健康研究中心，CSIRO，澳大利亚）；National Library of Medicine, National Institutes of Health, USA（国家医学图书馆，国立卫生研究院，美国）
ClinicalBench: Can LLMs Beat Traditional ML Models in Clinical Prediction?
Department of Computer Science Northwestern University Evanston USA（计算机科学系西北大学艾文斯顿美国）；Department of Computer Science University of Texas at Austin Austin USA（计算机科学系德克萨斯大学奥斯汀美国）；Boston Children's Hospital, Harvard Medical School Boston USA（波士顿儿童医院哈佛医学院波士顿美国）；Department of Computer Science Imperial College London London UK（计算机科学系伦敦帝国学院伦敦英国）；Department of Computer Science Ohio State University Columbus USA（计算机科学系俄亥俄州立大学哥伦布美国）；Massachusetts General Hospital, Harvard Medical School Boston USA（麻省总医院哈佛医学院波士顿美国）；Department of Preventive Medicine, Feinberg School of Medicine Northwestern University Chicago USA（预防医学系费因伯格医学院西北大学芝加哥美国）；Division of Computational Health Sciences, Department of Surgery University of Minnesota Minneapolis USA（计算健康科学部外科部明尼苏达大学明尼阿波利斯美国）；Department of Population Health Sciences, Weill Cornell Medicine Cornell University New York USA（流行病学与公共卫生系韦尔·科恩医学中心康奈尔大学纽约美国）；Department of Computer Science Emory University Atlanta USA（计算机科学系埃默里大学亚特兰大美国）；Northwestern University（西北大学）；University of Texas at Austin（德克萨斯大学奥斯汀）；Boston Children's Hospital, Harvard Medical School（波士顿儿童医院哈佛医学院）；Imperial College London（伦敦帝国学院）；Ohio State University（俄亥俄州立大学）；Massachusetts General Hospital, Harvard Medical School（麻省总医院哈佛医学院）；University of Minnesota（明尼苏达大学）；Cornell University（康奈尔大学）；Emory University（埃默里大学）
SUP-MIMIC: A Multi-Task Clinical Diagnosis Benchmark for Evaluating LLMs' Robustness to Contradictory Evidence
Beijing University of Technology（北京工业大学）；Beijing Institute of Technology（北京理工大学）；China-Japan Friendship Hospital（中日友好医院）；Dongbei University of Finance and Economics（东北财经大学）
Can Generic LLMs Help Analyze Child-adult Interactions Involving Children with Autism in Clinical Observation?
University of Southern California（南加州大学）；Boston University（波士顿大学）；Korea University（高丽大学）；University of California, San Francisco（加利福尼亚大学旧金山分校）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
DiReCT: Diagnostic Reasoning for Clinical Notes via Large Language Models
Premium Research Institute for Human Metaverse Medicine (WPI-PRIMe)（人类元宇宙医学高级研究机构（WPI-PRIMe））；Osaka University（大阪大学）；Department of Cardiology, The First Affiliated Hospital of Dalian Medical University（大连医科大学第一附属医院心内科）；Institute of Computing Technology, Chinese Academy of Science（中国科学院计算技术研究所）；D3 Center, Osaka University（大阪大学D3中心）；Agency for Science, Technology and Research (A*STAR)（科技研究局（A*STAR））
LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validation
College of Computer Science and Artificial Intelligence, Fudan University（计算机科学与人工智能学院，复旦大学）；Shanghai Key Lab of Intelligent Information Processing（上海智能信息处理重点实验室）；Shanghai AI Laboratory（上海人工智能实验室）；Northwestern University（西北大学）
Evaluating Clinical Competencies of Large Language Models with a General Practice Benchmark
The Sixth Affiliated Hospital of Sun Yat-sen University（中山大学第六附属医院）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东省人工智能与数字经济发展实验室（深圳））；Xinyi People’s Hospital（新一人民医院）；The Fifth Affiliated Hospital of Sun Yat-sen University（中山大学第五附属医院）；School of Public Health of Sun Yat-sen University（中山大学公共卫生学院）
PsychBench: A comprehensive and professional benchmark for evaluating the performance of LLM-assisted psychiatric clinical practice
Shanghai Jiao Tong University（上海交通大学）；Beijing Anding Hospital, Capital Medical University（首都医科大学附属北京安定医院）；Capital Medical University（首都医科大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
Benchmarking and Adapting On-Device LLMs for Clinical Decision Support
AI Collaborative Centre, University Health Network（人工智能协同中心，健康大学网络）；Princess Margaret Cancer Centre, University Health Network（彭菲癌症中心，健康大学网络）；Department of Electrical and Computer Engineering, University of Toronto（电气与计算机工程系，多伦多大学）；Division of Urology, Department of Surgery, St. Michael’s Hospital, Unity Health Toronto and University of Toronto（泌尿科，外科部，圣米歇尔医院，统一健康多伦多及多伦多大学）；Peter Munk Cardiac Centre, University Health Network（彼得·默克心脏中心，健康大学网络）；Department of Laboratory Medicine and Pathobiology and Department of Computer Science, University of Toronto（实验室医学与病理学系及计算机科学系，多伦多大学）；Vector Institute, Toronto（向量研究所，多伦多）
Can "AI" Be a Doctor? A Study of Empathy, Readability, and Alignment in Clinical LLMs
Department of Electrical Engineering and Information Technology（电子工程与信息科技系）；University of Naples Federico II（那不勒斯费德里科二世大学）；Department of Translational Medical Sciences（转化医学科学系）；University of Campania ”Luigi Vanvitelli”（坎帕尼亚“路易吉·范维蒂利”大学）；Department of Computer Science, McCormick School of Engineering and Applied Science（计算机科学系，麦科马克工程与应用科学学院）；Northwestern University（西北大学）
ClinBench-HPB: A Clinical Benchmark for Evaluating LLMs in Hepato-Pancreato-Biliary Diseases
The Hong Kong Polytechnic University（香港理工大学）；Shenzhen Institutes of Advanced Technology, CAS（深圳先进技术研究院）；Zhujiang Hospital, Southern Medical University（南方医科大学珠江医院）；The Key Laboratory of Biomedical Imaging Science and System, CAS（生物医学成像科学与系统重点实验室）
Same Patient, Different Words, Different Diagnosis? Evaluating Semantic Stability in Clinical LLMs
Department of Computer Science and Engineering, College of Engineering, Qatar University（卡塔尔大学计算机科学与工程系）；College of Science and Engineering, Hamad Bin Khalifa University (HBKU)（哈马德·本·卡伊夫大学（HBKU）理学院）；Primary Health Care Corporation (PHCC)（初级卫生保健公司）；Birmingham City University（伯明翰城市大学）
Leveraging Large Language Model as Simulated Patients for Clinical Education
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；School of Computer Science, Wuhan University（武汉大学计算机学院）；CureFun Co.（CureFun公司）
Large language models (LLMs) hold promise in clinical decision support but face major challenges in safety evaluation and effectiveness validation. We developed the Clinical Safety-Effectiveness Dual-Track Benchmark (CSEDB), a multidimensional framework built on clinical expert consensus, encompassing 30 criteria covering critical areas like critical illness recognition, guideline adherence, and medication safety, with weighted consequence measures. Thirty-two specialist physicians developed and reviewed 2,069 open-ended Q&A items aligned with these criteria, spanning 26 clinical departments t…
