Human organs constantly undergo anatomical changes due to a complex mix of short-term (e.g., heartbeat) and long-term (e.g., aging) factors. Evidently, prior knowledge of these factors will be beneficial when modeling their future state, i.e., via image generation. However, most of the medical image generation tasks only rely on the input from a single image, thus ignoring the sequential dependency even when longitudinal data is available. Sequence-aware deep generative models, where model input is a sequence of ordered and timestamped images, are still underexplored in the medical imaging domain that is featured by several unique challenges: 1) Sequences with various lengths; 2) Missing data or frame, and 3) High dimensionality. To this end, we propose a sequence-aware diffusion model (SADM) for the generation of longitudinal medical images. Recently, diffusion models have shown promising results in high-fidelity image generation. Our method extends this new technique by introducing a sequence-aware transformer as the conditional module in a diffusion model. The novel design enables learning longitudinal dependency even with missing data during training and allows autoregressive generation of a sequence of images during inference. Our extensive experiments on 3D longitudinal medical images demonstrate the effectiveness of SADM compared with baselines and alternative methods. The code is available at https://github.com/ubc-tea/SADM-Longitudinal-Medical-Image-Generation.
Temporal Flow Matching for Learning Spatio-Temporal Trajectories in 4D Longitudinal Medical Imaging
Division of Medical Image Computing, German Cancer Research Center(德国癌症研究中心医学影像计算部);HIDSS4Health - Helmholtz Information and Data Science School for Health(HIDSS4Health-海德堡信息与数据科学健康学校);Faculty of Mathematics and Computer Science, University of Heidelberg(海德堡大学数学与计算机科学学院);Pattern Analysis and Learning Group, Department of Radiation Oncology Heidelberg University Hospital(海德堡大学医院放射肿瘤科模式分析与学习组);Medical Faculty Heidelberg, University of Heidelberg(海德堡大学医学学院);Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital(海德堡大学医院放射肿瘤科模式分析与学习组)
4DLoG: Generative Modeling of Neurodegenerative Brain Anatomy with 4D Longitudinal Diffusion Model
Department of Electrical and Computer Engineering(电气与计算机工程系);Department of Radiology(放射学系);Department of Electrical and Computer Engineering, Department of Computer Science(电气与计算机工程系、计算机科学系)
Medical world models in healthcare: capabilities, evidence, and trustworthy clinical translation
National and Local Joint Engineering Laboratory of Computer Aided Design, School of Software Engineering, Dalian University(大连大学软件工程学院计算机辅助设计国家地方联合工程实验室);Department of Radiology, Xinhua Hospital Affiliated to Dalian University(大连大学附属新华医院放射科);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));University of California, San Francisco(加州大学旧金山分校);Yale University(耶鲁大学);The Hong Kong Polytechnic University(香港理工大学)
Medical Video Generation for Disease Progression Simulation
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Texas at Austin(德克萨斯大学奥斯汀分校);SambaNova System, Inc(SambaNova系统公司);Objective, Inc(Objective公司);Microsoft(微软公司);University of Virginia(弗吉尼亚大学);Shenzhen Nanshan People’s Hospital(深圳市南山区人民医院);Shenzhen Children’s Hospital(深圳市儿童医院)
Harnessing the power of longitudinal medical imaging for eye disease prognosis using Transformer-based sequence modeling
Weill Cornell Medicine(威尔康奈尔医学院);The University of Texas at Austin(德克萨斯大学奥斯汀分校);Washington University School of Medicine(华盛顿大学医学院);Columbia University(哥伦比亚大学);National Eye Institute(国家眼科研究所);National Institutes of Health (NIH)(美国国立卫生研究院);National Center for Biotechnology Information (NCBI)(国家生物技术信息中心);National Library of Medicine (NLM)(国家医学图书馆)
Saliency Guided Longitudinal Medical Visual Question Answering
Dept. of Computer Science and Engineering University of California, San Diego(计算机科学与工程系,加州大学圣地亚哥分校);Dept. of Radiology and Biomedical Imaging Yale University(放射学与生物医学成像系,耶鲁大学)
Deep Learning for Longitudinal Medical Imaging: A Scoping Review
Mass General Brigham(麻省总医院布莱根医疗体系);Harvard Medical School(哈佛医学院);Dana-Farber Cancer Institute(丹娜-法伯癌症研究所);Brigham and Women’s Hospital(布莱根妇女医院);University of California, San Francisco(加州大学旧金山分校);Inselspital, Bern University Hospital(伯尔尼大学医院( Inselspital ));University of Bern(伯尔尼大学)
Graph Representation Learning of Longitudinal Medical Imaging Trajectories for Treatment Response Prediction
Technical University of Munich(慕尼黑工业大学);TUM University Hospital Rechts der Isar(慕尼黑工业大学伊萨尔河右岸大学医院);Helmholtz Munich(慕尼黑亥姆霍兹中心);Universitat de Barcelona(巴塞罗那大学);Medical University of Vienna(维也纳医科大学);King’s College London(伦敦国王学院);Munich Center for Machine Learning(慕尼黑机器学习中心)
Is Complete Labeling Necessary? Understanding Active Learning in Longitudinal Medical Imaging
School of Computer Science, University College Dublin(都柏林大学计算机科学学院);Insight Centre for Data Analytics(数据分析研究中心);School of Medicine, University College Dublin(都柏林大学医学院);Department of Radiology, St Vincent’s University Hospital(圣文森大学医院放射科);Great Ormond Street Hospital for Children(儿童医院)
ImageFlowNet: Forecasting Multiscale Image-Level Trajectories of Disease Progression with Irregularly-Sampled Longitudinal Medical Images
Yale University(耶鲁大学);University of California, San Francisco(加州大学旧金山分校);Mila - Quebec AI Institute(魁北克AI研究所);Université de Montréal(蒙特利尔大学);McGill University(麦吉尔大学);Northern California Retina Vitreous Associates(北加州视网膜与玻璃体协会)
Stochastic Siamese MAE Pretraining for Longitudinal Medical Images
Institute of Artificial Intelligence, Center for Medical Data Science, Medical University of Vienna(人工智能研究所,医学数据科学中心,维也纳医科大学);Department of Ophthalmology and Optometry, Medical University of Vienna(眼科学与视光学系,维也纳医科大学);Ophthalmic Image Analysis Group (OPTIMA), Medical University of Vienna(眼科影像分析组(OPTIMA),维也纳医科大学);BioMedIA, Department of Computing, Imperial College London(BioMedIA,计算系,伦敦帝国理工学院);Chair for AI in Healthcare and Medicine, Technical University of Munich(医学与健康人工智能教授职位,慕尼黑技术大学);Moorfields National Institute for Health and Care Biomedical Research Centre, Moorfields Eye Hospital(莫尔菲尔兹国家健康与护理生物医学研究中心,莫尔菲尔兹眼科医院)
CRONOS: Continuous Time Reconstruction for 4D Medical Longitudinal Series
Division of Medical Image Computing, German Cancer Research Center(德国癌症研究所以的医学影像计算部);HIDSS4Health - Helmholtz Information and Data Science School for Health(Helmholtz健康信息与数据科学学校);Faculty of Mathematics and Computer Science, University of Heidelberg(海德堡大学数学与计算机科学学院);Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital(海德堡大学医院放射肿瘤学部模式分析与学习小组);Medical Faculty Heidelberg, University of Heidelberg(海德堡大学医学学院)
GLOMIA-Pro: A Generalizable Longitudinal Medical Image Analysis Framework for Disease Progression Prediction
School of Medical Technology, Beijing Institute of Technology(北京理工大学医学技术学院);Department of Thoracic Surgery, Sun Yat-sen University Cancer Center(中山大学肿瘤医院胸外科部门);Department of Radiology, The Fifth Affiliated Hospital of Sun Yat-sen University(中山大学第五附属医院放射科部门);Beijing Duanliu Medical Technology Co., Ltd.(北京杜流医疗科技有限公司)
Clinical Trajectory Alignment for Medical Vision-Language Pre-training
Institute of Intelligent Information Processing, Shanxi University(山西大学智能信息处理研究所);Alliance Manchester Business School, The University of Manchester(曼彻斯特大学联盟曼彻斯特商学院)
Stanford University(斯坦福大学);Stanford Medical Center(斯坦福医学中心);Stanford Health Care(斯坦福医疗保健系统);Stanford Center for Artificial Intelligence in Medicine and Imaging(斯坦福医学与影像人工智能中心);Prealize Health;Stanford School of Medicine(斯坦福医学院);Human-Centered Artificial Intelligence Institute, Stanford University(斯坦福大学以人为本人工智能研究所);Clinical Excellence Research Center, Stanford School of Medicine(斯坦福医学院临床卓越研究中心)
L-MAE: Longitudinal masked auto-encoder with time and severity-aware encoding for diabetic retinopathy progression prediction
LaTIM UMR 1101, Inserm(法国国家健康与医学研究院UMR 1101拉蒂姆研究所);University of Western Brittany(西布列塔尼大学);IMT Atlantique(大西洋电信学院);Lariboisière Hospital, AP-HP(巴黎公立医院集团拉里布瓦西耶尔医院);University Hospital of Brest(布雷斯特大学医院);INSERM U1227 Lymphocytes B et Autoimmunite (LBAI)(法国国家健康与医学研究院UMR 1227 B淋巴细胞与自身免疫研究所)
Context-aware deep learning using individualized prior information reduces false positives in disease risk prediction and longitudinal health assessment
New York University Grossman School of Medicine(纽约大学格罗斯曼医学院)
Deep Learning-Based Longitudinal Prediction of Childhood Myopia Progression Using Fundus Image Sequences and Baseline Refraction Data
Beijing Tongren Hospital, Capital Medical University(首都医科大学附属北京同仁医院);School of Instrumentation and Optoelectronic Engineering, Beihang University(北京航空航天大学仪器科学与光电工程学院);School of Mechanical Engineering, Beihang University(北京航空航天大学机械工程学院);University of Cambridge(剑桥大学);University College London(伦敦大学学院);The Chinese University of Hong Kong(香港中文大学);Thorough Future Inc.(透景未来科技有限公司);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学)
MedDIFT: Multi-Scale Diffusion-Based Correspondence in 3D Medical Imaging
1 School of Computation, Information;Technology, Technical University of Munich 2 Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich 3 Munich Center for Machine Learning (MCML) 4 Institute for Diagnostic;Interventional Radiology, Klinkum Rechts der Isar 5 School of Biomedical Engineering \& Imaging Sciences, King’s College London
Self-supervised Lesion Change Detection and Localisation in Longitudinal Multiple Sclerosis Brain Imaging
Flinders University(弗林德斯大学);Flinders Medical Centre(弗林德斯医疗中心);Dr Jones & Partners Medical Imaging(琼斯博士及合伙人医疗影像中心);University of Oxford(牛津大学);University of Adelaide(阿德莱德大学)
MedAtlas: Evaluating LLMs for Multi-Round, Multi-Task Medical Reasoning Across Diverse Imaging Modalities and Clinical Text
University of Science and Technology of China(中国科学技术大学);Eastern Institute of Technology(东方理工大学);Shanghai Jiao Tong University(上海交通大学);Hefei Comprehensive National Science Center(合肥综合性国家科学中心)