MedPMC: A Systematic Framework for Scaling High-Fidelity Medical Multimodal Data for Foundation Models
Yale University(耶鲁大学);Korea University(韩国大学);Washington University in St. Louis(圣路易斯华盛顿大学);The University of Queensland(昆士兰大学);The University of Texas Health Science Center at Houston(德克萨斯大学休斯顿健康科学中心);University of Washington(华盛顿大学);Microsoft Research(微软研究院)
Hyunjae Kim, Dain Kim, Pan Xiao, Serina S. Applebaum, Younjoon Chung, Xuguang Ai, Yu Yin, Roy Jiang, Yuexi Du, Yawen Wei, Yiming Kong, Tuo Guo, Zhiyuan Cao, Men… 展开作者
Hyunjae Kim, Dain Kim, Pan Xiao, Serina S. Applebaum, Younjoon Chung, Xuguang Ai, Yu Yin, Roy Jiang, Yuexi Du, Yawen Wei, Yiming Kong, Tuo Guo, Zhiyuan Cao, Mengmeng Du, Yuelei Fu, Yan Hu, Rui Shi, Gui Yang, Kevin W. Jin, Yuntian Liu, Yuxuan Tian, Jonathan Marquez, Zhen Chen, Sheng Zhang, Hoifung Poon, Hua Xu, Jaewoo Kang, Qingyu Chen
Medicine is inherently multimodal, requiring clinicians to synthesize information across diverse data streams. Yet the development of multimodal foundation models is constrained by limited access to large-scale, high-quality clinical data. Although PubMed Central (PMC) offers a complementary source of expert-authored image-text data, existing PMC-derived resources remain limited in fidelity, reproducibility, and clinical validation. We introduce MedPMC, an automated, continuously updatable framework that transforms permissively licensed literature into high-fidelity infrastructure for medical multimodal models. Applied to 6.1 million PMC articles, MedPMC curated 11 million medical image-text pairs. Component evaluations showed strong performance for initial screening (F1 = 93.2), multi-panel figure detection (F1 = 96.5), figure separation (mAP = 89.8), caption separation and alignment (F1 = 81.4; ROUGE-L = 85.3), and medical figure classification (F1 = 96.5). Manual review by five annotators, three with medical training, found 95.3% of MedPMC images medically relevant, versus 19.7% in a prior PMC-derived dataset. Across 26 benchmarks spanning 11 specialties, a MedPMC-trained CLIP-style model improved average zero-shot AUC by 7.1 percentage points over the strongest architecture-matched biomedical CLIP baseline despite using fewer than half as many image-text pairs. As the vision encoder in a multimodal large language model, it improved medical visual question-answering by 1.9 and 16.9 percentage points across two benchmarks. In 10,524 Yale New Haven Health System dermatology photographs, it improved morphology-to-image retrieval Recall@5 by 11.7 percentage points. These findings show that high-fidelity literature curation strengthens medical multimodal foundation models across benchmark and clinical settings. We publicly release the framework, corpus, benchmarks, and pretrained models.
Evaluating Reasoning Faithfulness in Medical Vision-Language Models using Multimodal Perturbations
Technical University of Munich (TUM)(慕尼黑技术大学);TUM University Hospital(慕尼黑技术大学医院);German Heart Center TUM University Hospital(慕尼黑技术大学医院德国心脏中心);Department of Radiology(放射科);Klinikum rechts der Isar TUM University Hospital(慕尼黑技术大学医院右岸诊所);Uniklinik RWTH Aachen(亚琛工业大学医院);HOPPR IL USA(HOPPR美国);University of Oxford(牛津大学);Imperial College London(伦敦帝国学院)
OpenMedReason: Scientific Reasoning Supervision for Medical Vision-Language Models
York University(约克大学);Vector Institute(向量研究所);University of British Columbia(不列颠哥伦比亚大学);University of Toronto(多伦多大学);Unity Health Toronto / St. Michael’s Hospital(多伦多联合健康/圣迈克尔医院);University Health Network(大学健康网络);Arc Institute(弧研究所);Queen's University(女王大学)
Parameter-Efficient VLMs for Gastrointestinal Endoscopy: Medical Image Generation and Clinical Visual Question Answering
Computer Science Department, Morgan State University(莫尔甘州大学计算机科学系);International Organization for Migration (IOM)(国际移民组织);Electrical & Computer Engineering Department, Morgan State University(莫尔甘州大学电气与计算机工程系)
Any-to-Any Vision-Language Model for Multimodal X-ray Imaging and Radiological Report Generation
Department of Diagnostics and Intervention, Biomedical Engineering and Radiation Physics, Umeå University(诊断与介入部门,生物医学工程与放射物理,乌梅大学);College of Computer Science and Software Engineering, Shenzhen University(计算机科学与软件工程学院,深圳大学)
Linear Algebraic Approaches to Neuroimaging Data Compression: A Comparative Analysis of Matrix and Tensor Decomposition Methods for High-Dimensional Medical Images
Few-Shot Medical Image Segmentation with High-Fidelity Prototypes
University of Shanghai for Science and Technology(上海理工大学);Universität Hamburg(汉堡大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Electronic Science and Technology of China(电子科技大学);University of Surrey(萨里大学)
Generative Drifting for Conditional Medical Image Generation
Department of Orthodontics and Orofacial Orthopedics, Friedrich-Alexander-University Erlangen-Nuremberg(口腔正畸与面部骨科系,弗里德里希-艾萨克-埃尔兰-纽伦堡大学);Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-University Erlangen-Nuremberg(生物医学工程人工智能系,弗里德里希-艾萨克-埃尔兰-纽伦堡大学);Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg(模式识别实验室,弗里德里希-艾萨克-埃尔兰-纽伦堡大学);Department of Biomedical Engineering, Sun-Yat-sen University(生物医学工程系,孙中山大学)
Dino U-Net: Exploiting High-Fidelity Dense Features from Foundation Models for Medical Image Segmentation
School of Biomedical Engineering (Suzhou), Division of Life Science and Medicine, University of Science and Technology of China, Hefei, China(生物医学工程学院(苏州),生命科学与医学系,中国科学技术大学,合肥,中国);Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China(苏州生物医学工程与技术研究所,中国科学院,苏州,中国);Shanghai Innovation Institute, Shanghai, China(上海创新研究院,上海,中国);Medical School of Tianjin University, Tianjin, China(天津大学医学院,天津,中国);Jinan Guoke Medical and Technology Development Co., Ltd., Pharmaceutical Valley New Drug Creation Platform, Jinan, China(济南国科医药科技发展有限公司,药谷新药创制平台,济南,中国)
Sparse Autoencoders for Interpretable Medical Image Representation Learning
Stanford Center for Artificial Intelligence in Medicine and Imaging(斯坦福大学人工智能医学与成像中心);Department of Radiology, Stanford University(斯坦福大学放射科);Chair of AI in Healthcare and Medicine, Technical University of Munich(慕尼黑技术大学人工智能医疗与医学主任);TUM University Hospital, Munich, Germany(慕尼黑技术大学医院)
EchoNet-Synthetic: Privacy-preserving Video Generation for Safe Medical Data Sharing
UKRI CDT in AI for Healthcare, Imperial College London(英国帝国理工学院UKRI医疗保健人工智能博士培训中心);Department of Computing, Imperial College London(帝国理工学院计算系);Friedrich–Alexander University Erlangen–Nürnberg(埃尔朗根-纽伦堡弗里德里希-亚历山大大学);School of BMEIS, King’s College London(伦敦国王学院BMEIS学院);Guy’s and St Thomas’ NHS Foundation Trust(盖伊和圣托马斯NHS基金会信托);Ultromics Ltd(Ultromics有限公司);John Radcliffe Hospital, Cardiovascular Clinical Research Facility(约翰拉德克利夫医院心血管临床研究中心)
Domain-Specific Latent Representations Improve the Fidelity of Diffusion-Based Medical Image Super-Resolution
Massachusetts Institute of Technology(麻省理工学院);Department of Biomedical Engineering, Mbarara University of Science and Technology(姆巴拉大学科学与技术学院生物医学工程系);Johns Hopkins University(约翰霍普金斯大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Makerere University(Makerere大学);Federal University of São Paulo(圣保罗联邦大学);Technical University of Applied Sciences Lübeck (TH Lübeck)(吕贝克应用技术大学);Harvard T.H. Chan School of Public Health(哈佛大学T.H. Chan公共卫生学院)
VolDiT: Controllable Volumetric Medical Image Synthesis with Diffusion Transformers
Institute for Artificial Intelligence in Cardiovascular Medicine, Medical Faculty of Heidelberg University, Heidelberg University(海德堡大学医学院心血管医学人工智能研究所);Department of Cardiology, Angiology, Pneumology, Heidelberg University Hospital(海德堡大学医院心脏病学、血管学、肺病学系);DZHK (German Centre for Cardiovascular Research), Partner Site Heidelberg/Mannheim(德国心血管研究中心(DZHK)海德堡/曼海姆合作站点);University Medical Center of the Johannes Gutenberg-University Mainz(美因茨约翰内斯·古腾堡大学医学中心)
Fuse4Seg: Image Fusion for Multi-Modal Medical Segmentation via Bi-level Optimization
Northwestern University(西北大学);Northeastern University(东北大学);South China University of Technology(华南理工大学);Hong Kong Baptist University(香港 Baptist大学);Beijing Normal - Hong Kong Baptist University(北京师范大学-香港 Baptist大学)
GAN-GA: A Generative Model based on Genetic Algorithm for Medical Image Generation
Faculty of Computers and Artificial Intelligence, Helwan University(赫勒万大学计算机与人工智能学院);TachyHealth(TachyHealth公司);Information Technology and Computer Science School, Nile University(尼罗大学信息技术与计算机科学学院)
Balancing Fidelity, Utility, and Privacy in Synthetic Cardiac MRI Generation: A Comparative Study
University of Peradeniya, Sri Lanka(斯里兰卡大学);University of Maryland, College Park, USA(美国马里兰大学 College Park 分校);Tulane University School of Medicine, USA(美国 Tulane 大学医学院);Simula Research Laboratory, Norway(挪威 Simula 研究实验室);SimulaMet, Norway(挪威 SimulaMet)
PPDM: Pixel Puzzling Diffusion Model for Speed and Memory Efficient Volumetric Medical Image Translation
Department of Radiology, Northwestern University(西北大学放射学系);Department of Biomedical Engineering, Yale University(耶鲁大学生物医学工程系);Department of Radiology and Biomedical Imaging, Yale School of Medicine(耶鲁医学院放射学与生物医学影像系)
Deformation-aware GAN for Medical Image Synthesis with Substantially Misaligned Pairs
Australian e-Health Research Centre, CSIRO(澳大利亚电子健康研究中心(联邦科学与工业研究组织));Liverpool and Macarthur Cancer Therapy Centres and Ingham Institute(利物浦与麦卡瑟癌症治疗中心及英厄姆研究所);Child Health Research Centre, University of Queensland(昆士兰大学儿童健康研究中心);Queensland Children’s Hospital(昆士兰儿童医院);Data61, CSIRO(Data61(联邦科学与工业研究组织下属机构));Siemens Healthcare Pty Ltd(西门子医疗私人有限公司);Siemens Healthineers AG(西门子医疗保健股份公司)
Noise-Consistent Siamese-Diffusion for Medical Image Synthesis and Segmentation
National University of Singapore(国立新加坡大学);National University of Singapore Suzhou Research Institute(国立新加坡大学苏州研究院);Wenzhou-Kean University(温州-凯恩大学);Soochow University(苏州大学);City University of Hong Kong(香港城市大学)
From Redaction to Restoration: Deep Learning for Medical Image Anonymization and Reconstruction
Center for Artificial Intelligence, BCVI, Northwestern Medicine(西北医学BCVI人工智能中心);Department of Electrical and Computer Engineering, Northwestern University(西北大学电气与计算机工程系);Department of Surgery, Northwestern University(西北大学外科学系);Xtasis Inc.(Xtasis公司);Medtronic(美敦力公司);Departments of Radiology and Clinical Neurosciences, University of Calgary(卡尔加里大学放射学与临床神经科学系)
HiFi-Syn: Hierarchical Granularity Discrimination for High-Fidelity Synthesis of MR Images with Structure Preservation
Ruijin Hospital, Shanghai Jiao Tong University School of Medicine(上海交通大学医学院附属瑞金医院);College of Health Science and Technology, Shanghai Jiao Tong University School of Medicine(上海交通大学医学院健康科学技术学院);Clinical Neuroscience Center, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine(上海交通大学医学院附属瑞金医院临床神经科学中心);National Engineering Research Center of Advanced Magnetic Resonance Technologies for Diagnosis and Therapy, Shanghai Jiao Tong University(上海交通大学先进磁共振诊断与治疗技术国家工程研究中心);Ping An Technology (Shenzhen) Co., Ltd.(平安科技(深圳)有限公司)
Low-Level Dataset Distillation for Medical Image Enhancement
School of Cyber Science and Engineering, Sichuan University(四川大学计算机科学与工程学院);The Chinese University of Hong Kong(香港中文大学);Agency for Science, Technology and Research (A*STAR)(科技研究局)