Does Biomedical Training Lead to Better Medical Performance?
Institute for AI in Medicine, University Hospital Essen（人工智能医学研究所，埃森大学医院）；Microsoft Healthcare & Life Sciences（微软医疗与生命科学）；NVIDIA（NVIDIA公司）；Cancer Research Center Cologne Essen (CCCE)（科隆埃森癌症研究中心（CCCE））；German Cancer Consortium (DKTK, Partner site Essen)（德国癌症联盟（DKTK，埃森分部））；Department of Physics of TU Dortmund（多特蒙德技术大学物理系）
Amin Dada, Marie Bauer, Amanda Butler Contreras, Osman Alperen Koraş, Constantin Marc Seibold, Kaleb E Smith, Jens Kleesiek
备注：
Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics (GEM)
Large Language Models (LLMs) are expected to significantly contribute to patient care, diagnostics, and administrative processes. Emerging biomedical LLMs aim to address healthcare-specific challenges, including privacy demands and computational constraints. Assessing the models' suitability for this sensitive application area is of the utmost importance. However, biomedical training has not been systematically evaluated on medical tasks. This study investigates the effect of biomedical training in the context of six practical medical tasks evaluating $25$ models. In contrast to previous evaluations, our results reveal a performance decline in nine out of twelve biomedical models after fine-tuning, particularly on tasks involving hallucinations, ICD10 coding, and instruction adherence. General-domain models like Meta-Llama-3.1-70B-Instruct outperformed their biomedical counterparts, indicating a trade-off between domain-specific fine-tuning and general medical task performance. We open-source all evaluation scripts and datasets at https://github.com/TIO-IKIM/CLUE to support further research in this critical area.
Doctor Sun: A Bilingual Multimodal Large Language Model for Biomedical AI
Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education East China University of Science and Technology（能源化工过程智能制造重点实验室，东华大学）；Research Institute of Intelligent Control and Systems Harbin Institute of Technology（智能控制与系统研究室，哈尔滨工业大学）；Department of Emergency Medicine, Sir Run Run Shaw Hospital Zhejiang University School of Medicine（浙江大学医学院急诊医学科）；Provincial Key Laboratory of Precise Diagnosis Treatment of Abdominal Infection, Sir Run Run Shaw Hospital Zhejiang University School of Medicine（腹部感染精准诊断治疗省级重点实验室，浙江大学医学院）；School of Medicine Shaoxing University（绍兴大学医学院）
Doctor Sun是一种双语多模态大语言模型，通过整合预训练视觉编码器和医学LLM，提升生物医学多模态任务的性能，并提供SunMed-VL数据集支持研究进展。
BioPars: A Pretrained Biomedical Large Language Model for Persian Biomedical Text Mining
Department of Computer Science, Faculty of Education, University of Kufa（计算机科学系，教育学院，科菲大学）；Department of Computer Science, Florida State University（计算机科学系，佛罗里达州立大学）；Department of Computer Engineering, Velayat University（计算机工程系，瓦莱yat大学）；Department of Mechanical Engineering, Amirkabir University of Technology（机械工程系，阿米卡比尔技术大学）；Department of Computer Engineering, Ferdowsi University of Mashhad（计算机工程系，法尔德大学）
From Compound Figures to Medical Multi-image Reasoning: Scaling Multimodal Large Language Models with Biomedical Literature
Department of Biomedical Informatics and Data Science, Yale School of Medicine, Yale University, New Haven, CT 06510, USA（耶鲁医学院生物医学信息学与数据科学系，耶鲁大学，新 Haven，CT 06510，USA）；School of Medicine, University of Puerto Rico, San Juan, PR 00921, USA（波多黎各大学医学院，波多黎各，San Juan，PR 00921，USA）
Hierarchical Dual-Strategy Unlearning for Biomedical and Healthcare Intelligence Using Imperfect and Privacy-Sensitive Medical Data
Department of Neurosurgery, Qilu Hospital of Shandong University and Institute of Brain and Brain-Inspired Science（齐鲁医院山东大学神经外科和脑科学与脑启发科学研究院）；Department of Neurosurgery, The Affiliated Hospital of Qingdao University（青岛大学附属医院神经外科）；Department of Neurosurgery, Peking Union Medical College Hospital（北京地坛医院神经外科）；School of Software and Microelectronics, Peking University（北京大学软件与微电子学院）；College of Artificial Intelligence, Dalian Maritime University（大连海事大学人工智能学院）；University of Colorado Boulder（科罗拉多大学波尔德分校）；Zhengzhou University（郑州大学）；The University of Hong Kong（香港大学）；Georgia Institute of Technology（佐治亚理工学院）
Empowering Medical Equipment Sustainability in Low-Resource Settings: An AI-Powered Diagnostic and Support Platform for Biomedical Technicians
Carnegie Mellon University Africa（卡内基梅隆大学非洲分校）；Medical Artificial Intelligence Laboratory（医学人工智能实验室）；University of Pennsylvania（宾夕法尼亚大学）；McGill University（麦吉尔大学）
Medical Foundation Models are Susceptible to Targeted Misinformation Attacks
University Hospital Aachen（亚琛大学医院）；University Medical Center of the Johannes Gutenberg-University（约翰内斯·古腾堡大学医学中心）；University Medicine Essen（埃森大学医学部）；Ocumeda GmbH（Ocumeda有限公司）；Charité - Universitätsmedizin Berlin（柏林夏里特医学院）；Freie Universität Berlin（柏林自由大学）；Humboldt Universität zu Berlin（柏林洪堡大学）；Berlin Institute of Health at Charité - Universitätsmedizin Berlin（柏林夏里特医学院柏林健康研究所）；Technical University Dresden（德累斯顿工业大学）；University Hospital Dresden（德累斯顿大学医院）；University Hospital Heidelberg（海德堡大学医院）
Comprehensive Study on German Language Models for Clinical and Biomedical Text Understanding
University of Applied Sciences and Arts Dortmund（多特蒙德应用科学与艺术大学）；University Hospital Essen（埃森大学医院）；Institute for AI in Medicine (IKIM)（医学人工智能研究所）；University Medicine Essen（埃森大学医学中心）；FernUniversität in Hagen（哈根函授大学）；NVIDIA（英伟达公司）；College of Medicine, University of Florida（佛罗里达大学医学院）；University of Florida Health Cancer Center（佛罗里达大学健康癌症中心）；University of Marburg（马尔堡大学）；University of Mannheim（曼海姆大学）；TU Dortmund（多特蒙德工业大学）
Biomedical Visual Instruction Tuning with Clinician Preference Alignment
Stanford University（斯坦福大学）；Emory University（埃默里大学）；University of California, Berkeley（加州大学伯克利分校）；University of Washington（华盛顿大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
Can Large Language Models Replace Data Scientists in Biomedical Research?
School of Computing and Data Science, University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校计算与数据科学学院）；Bioinformatics Center, Institute for Chemical Research, Kyoto University（京都大学化学研究所生物信息中心）；Institute of Scientific and Industrial Research, Osaka University（大阪大学科学与工业研究所）；Carle Illinois College of Medicine, University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校卡尔伊利诺伊医学院）
Utilizing Large Language Models for Zero-Shot Medical Ontology Extension from Clinical Notes
Department of Computer Science（计算机科学系）；Emory University（埃默里大学）；College of Public Health（公共卫生学院）；Temple University（temple大学）；School of Information（信息学院）；Florida State University（佛罗里达州立大学）；Hubert Department of Global Health（全球健康部）；Nell Hodgson Woodruff School of Nursing（Nell Hodgson Woodruff护理学院）
Does Bigger Mean Better? Comparitive Analysis of CNNs and Biomedical Vision Language Modles in Medical Diagnosis
University of Texas at Dallas（德克萨斯大学达拉斯分校）；Independent Researcher（独立研究者）；Duke University（杜克大学）；University of Michigan Ann Arbor（密歇根大学安娜堡分校）；Georgia Institute of Technology（佐治亚理工学院）；University of California, Irvine（加州大学 Irvine 分校）
Knowledge Hierarchy Guided Biological-Medical Dataset Distillation for Domain LLM Training
Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）；University of Chinese Academy of Sciences（中国科学院大学）；Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences（中国科学院大学杭州高等研究院）
Biomedical Nested NER with Large Language Model and UMLS Heuristics
Georgia Institute of Technology（佐治亚理工学院）；Peoples' Friendship University of Russia (RUDN University)（俄罗斯人民友谊大学（RUDN大学））；Joint Institute for Nuclear Research（联合核研究所）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；University of Skövde（舍夫德大学）
Holistic Evaluation of GPT-4V for Biomedical Imaging
University of Georgia（佐治亚大学）；Beijing Jiaotong University（北京交通大学）；Northwestern Polytechnical University（西北工业大学）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）；University of Electronic Science and Technology of China（电子科技大学）
Department of Computer Science, University of Illinois, Chicago, IL, 60607, USA（伊利诺伊大学芝加哥分校计算机科学系）；Indian Institute of Technology, Varanasi, India（印度班加罗尔理工学院）；Department of Computer Science & Engineering, Indraprastha Institute of Information Technology, Delhi, New Delhi, 110020, India（印度德里印度信息技术研究所计算机科学与工程系）
Biomedical Large Languages Models Seem not to be Superior to Generalist Models on Unseen Medical Data
Charité - Universitätsmedizin Berlin（柏林夏里特医学院）；Freie Universität Berlin（柏林自由大学）；Humboldt Universität zu Berlin（柏林洪堡大学）；Athinoula A. Martinos Center for Biomedical Imaging（雅典娜·马蒂诺斯生物医学成像中心）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Institute for AI in Medicine (IKIM)（医学人工智能研究所）；University Hospital Essen (AöR)（埃森大学医院）；Technical University of Munich（慕尼黑工业大学）；University Hospital Aachen（亚琛大学医院）；Cancer Research Center Cologne Essen (CCCE)（科隆埃森癌症研究中心）；West German Cancer Center Essen（西德埃森癌症中心）；German Cancer Consortium (DKTK, Partner site Essen)（德国癌症联盟（埃森合作站点））；TU Dortmund（多特蒙德工业大学）；Bakar Computational Health Sciences Institute（巴卡尔计算健康科学研究所）；University of California, San Francisco（加州大学旧金山分校）
King’s College London（伦敦国王学院）；University College London（伦敦大学学院）；NIHR Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London（英国国立卫生研究院生物医学研究中心（位于南伦敦和莫兹利 NHS 基金会信托与伦敦国王学院））
电子健康记录（EHR）等生物医学文档包含大量非结构化格式的信息。EHR中的数据是记录临床叙述和决策的非常有价值的资源，但是虽然文本可以很容易地被人类医生理解，但在研究和临床应用中使用却具有挑战性。为了揭示生物医学文档的潜力，我们需要提取和结构化它们所包含的信息。当前的任务是命名实体识别和链接（NER+L）。实体的数量，词语的模糊性，重叠和嵌套使得生物医学领域比许多其他领域更加困难。为了克服这些困难，我们开发了医学概念注释工具（MedCAT），这是NER+L的开源无监督方法。MedCAT使用无监督机器学习来消除实体的歧义。它在MIMIC-III（一个免费访问的重症监护数据库）和MedMentions（由统一医学语言系统注释的生物医学论文）上得到了验证。在NER+L的情况下，与现有工具的比较表明，MedCAT仅通过无监督学习改善了之前的最佳效果（疾病检测的F1=0.848 vs 0.691;一般概念检测的F1=0.710 vs 0.222）。MedCAT学习的向量嵌入的定性分析表明，它捕获了EHR（MIMIC-III）中的潜在医学知识。无监督学习可以提高大规模实体抽取的性能，但在处理少量实体和小数据集时存在一定的局限性。在这种情况下，选项是监督学习或主动学习，两者都通过MedCATtrainer扩展在MedCAT中得到支持。我们的方法可以检测和链接数百万种不同的生物医学概念，具有最先进…
