Enhancing Spatiotemporal Disease Progression Models via Latent Diffusion and Prior Knowledge
Dept. of Math and Computer Science, University of Catania, Italy（数学与计算机科学系，卡塔尼亚大学，意大利）；Centre for Medical Image Computing, University College London, UK（医学影像计算中心，伦敦大学学院，英国）；School of Physics, Engineering and CS, University of Hertfordshire, UK（物理、工程与计算机科学学院，赫特福德大学，英国）
在这项工作中，我们介绍了Brain Latent Progression (BrLP)，一种基于潜在扩散的新型时空疾病进展模型。BrLP旨在预测个体层面3D脑部MRI上的疾病演变。为该任务开发的现有深度生成模型主要是数据驱动的，在学习疾病进展方面面临挑战。BrLP通过整合来自疾病模型的先验知识来解决这些挑战，以提高预测的准确性。为此，我们提出整合一个辅助模型，该模型推断各脑区的体积变化。此外，我们引入了Latent Average Stabilization (LAS)，一种提高预测进展时空一致性的新技术。BrLP在一个包含2,805名受试者的11,730张T1加权脑部MRI的大型数据集上进行了训练和评估，这些数据来自三项公开可用的阿尔茨海默病(AD)纵向研究。在我们的实验中，我们将BrLP生成的MRI扫描与受试者可用的实际随访MRI进行了比较，包括横断面和纵向设置。BrLP相比现有方法显示出显著改进，在AD相关脑区的体积准确性提高了22%，与真实扫描的图像相似度提高了43%。BrLP生成受试者层面条件化3D扫描的能力，以及整合先验知识以提高准确性的新颖性，代表了疾病进展建模的重大进步，为精准医学开辟了新途径。BrLP的代码可在以下链接获取：https://github.com/LemuelPuglisi/BrLP。
英文摘要：
In this work, we introduce Brain Latent Progression (BrLP), a novel spatiotemporal disease progression model based on latent diffusion. BrLP is designed to predict the evolution of diseases at the individual level on 3D brain MRIs. Existing deep generative models developed for this task are primarily data-driven and face challenges in learning disease progressions. BrLP addresses these challenges by incorporating prior knowledge from disease models to enhance the accuracy of predictions. To implement this, we propose to integrate an auxiliary model that infers volumetric changes in various brain regions. Additionally, we introduce Latent Average Stabilization (LAS), a novel technique to improve spatiotemporal consistency of the predicted progression. BrLP is trained and evaluated on a large dataset comprising 11,730 T1-weighted brain MRIs from 2,805 subjects, collected from three publicly available, longitudinal Alzheimer's Disease (AD) studies. In our experiments, we compare the MRI scans generated by BrLP with the actual follow-up MRIs available from the subjects, in both cross-sectional and longitudinal settings. BrLP demonstrates significant improvements over existing methods, with an increase of 22% in volumetric accuracy across AD-related brain regions and 43% in image similarity to the ground-truth scans. The ability of BrLP to generate conditioned 3D scans at the subject level, along with the novelty of integrating prior knowledge to enhance accuracy, represents a significant advancement in disease progression modeling, opening new avenues for precision medicine. The code of BrLP is available at the following link: https://github.com/LemuelPuglisi/BrLP.
Brain Latent Progression: Individual-based Spatiotemporal Disease Progression on 3D Brain MRIs via Latent Diffusion
Dept. of Math and Computer Science, University of Catania（数学与计算机科学系，卡塔尼亚大学）；Centre for Medical Image Computing, University College London（医学影像计算中心，伦敦大学学院）；MIFT Department, University of Messina（MIFT系，梅斯ina大学）
提出 BrLP 时空模型，通过在小型潜在空间运行、整合受试者元数据与疾病动力学先验，并引入 LAS 算法，实现了 3D 脑部 MRI 个体化疾病进展的高精度预测。
Medical Video Generation for Disease Progression Simulation
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；SambaNova System, Inc（SambaNova系统公司）；Objective, Inc（Objective公司）；Microsoft（微软公司）；University of Virginia（弗吉尼亚大学）；Shenzhen Nanshan People’s Hospital（深圳市南山区人民医院）；Shenzhen Children’s Hospital（深圳市儿童医院）
Unlike natural images, medical images often have intrinsic characteristics that can be leveraged for neural network learning. For example, images that belong to different stages of a disease may continuously follow a certain progression pattern. In this paper, we propose a novel method that leverages disease progression learning for medical image recognition. In our method, sequences of images ordered by disease stages are learned by a neural network that consists of a shared vision model for feature extraction and a long short-term memory network for the learning of stage sequences. Auxiliary…
Centre for Medical Image Computing（医学图像计算中心）；Department of Computer Science, University College London（伦敦大学学院计算机科学系）
Simulating images representative of neurodegenerative diseases is important for predicting patient outcomes and for validation of computational models of disease progression. This capability is valuable for secondary prevention clinical trials where outcomes and screening criteria involve neuroimaging. Traditional computational methods are limited by imposing a parametric model for atrophy and are extremely resource-demanding. Recent advances in deep learning have yielded data-driven models for longitudinal studies (e.g., face ageing) that are capable of generating synthetic images in real-tim…
TransFair: Transferring Fairness from Ocular Disease Classification to Progression Prediction
School of Computing and Informatics, University of Louisiana at Lafayette（路易斯安那大学拉斐特分校计算与信息学院）；Informatics Research Institute, University of Louisiana at Lafayette（路易斯安那大学拉斐特分校信息学研究所）；College of Engineering and Computer Science, Florida Atlantic University（佛罗里达大西洋大学工程与计算机科学学院）；Harvard Medical School（哈佛医学院）
Erasmus MC（伊拉斯姆斯大学医学中心）；Delft University of Technology（代尔夫特理工大学）
The event-based model (EBM) for data-driven disease progression modeling estimates the sequence in which biomarkers for a disease become abnormal. This helps in understanding the dynamics of disease progression and facilitates early diagnosis by staging patients on a disease progression timeline. Existing EBM methods are all generative in nature. In this work we propose a novel discriminative approach to EBM, which is shown to be more accurate as well as computationally more efficient than existing state-of-the art EBM methods. The method first estimates for each subject an approximate orderin…
Peking University（北京大学）；Microsoft Research Asia（微软亚洲研究院）；Capital Medical University（首都医科大学）；Deepwise AI Lab（深睿医疗AI实验室）；Beijing Stars Universal Technology Co Ltd（北京星辰万联科技有限公司）
Forecasting Parapapillary atrophy (PPA), i.e., a symptom related to most irreversible eye diseases, provides an alarm for implementing an intervention to slow down the disease progression at early stage. A key question for this forecast is: how to fully utilize the historical data (e.g., retinal image) up to the current stage for future disease prediction? In this paper, we provide an answer with a novel framework, namely \textbf{D}isease \textbf{F}orecast via \textbf{P}rogression \textbf{L}earning (\textbf{DFPL}), which exploits the irreversibility prior (i.e., cannot be reversed once diagnos…
Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；Indian Institute of Technology Madras（印度理工学院马德拉斯分校）
Purpose: A core component of advancing cancer treatment research is assessing response to therapy. Doing so by hand, for example as per RECIST or RANO criteria, is tedious, time-consuming, and can miss important tumor response information; most notably, they exclude non-target lesions. We wish to assess change in a holistic fashion that includes all lesions, obtaining simple, informative, and automated assessments of tumor progression or regression. Due to often low patient enrolments in clinical trials, we wish to make response assessments with small training sets. Deep neuroevolution (DNE) c…
Learning Patient-Specific Disease Dynamics with Latent Flow Matching for Longitudinal Imaging Generation
University of Cambridge（剑桥大学）；Nanjing First Hospital（南京第一医院）；Nanjing Medical University（南京医科大学）；Johns Hopkins University（约翰霍普金斯大学）；University of Dundee（邓迪大学）
Align-cDAE: Alzheimer's Disease Progression Modeling with Attention-Aligned Conditional Diffusion Auto-Encoder
Department of Electrical Engineering, Indian Institute of Technology Madras, Chennai, India（电子工程系，印度理工学院马德拉斯，钦奈，印度）；Sudha Gopalakrishnan Brain Centre, Indian Institute of Technology Madras, Chennai, India（苏达·戈帕拉克里希南脑中心，印度理工学院马德拉斯，钦奈，印度）；Department of Electrical Engineering, Indian Institute of Technology Madras Chennai, India（电子工程系，印度理工学院马德拉斯钦奈，印度）
We propose a method to predict the subject-specific longitudinal progression of brain structures extracted from baseline MRI, and evaluate its performance on Alzheimer's disease data. The disease progression is modeled as a trajectory on a group of diffeomorphisms in the context of large deformation diffeomorphic metric mapping (LDDMM). We first exhibit the limited predictive abilities of geodesic regression extrapolation on this group. Building on the recent concept of parallel curves in shape manifolds, we then introduce a second predictive protocol which personalizes previously learned traj…
GLOMIA-Pro: A Generalizable Longitudinal Medical Image Analysis Framework for Disease Progression Prediction
School of Medical Technology, Beijing Institute of Technology（北京理工大学医学技术学院）；Department of Thoracic Surgery, Sun Yat-sen University Cancer Center（中山大学肿瘤医院胸外科部门）；Department of Radiology, The Fifth Affiliated Hospital of Sun Yat-sen University（中山大学第五附属医院放射科部门）；Beijing Duanliu Medical Technology Co., Ltd.（北京杜流医疗科技有限公司）
Clinical outcome or severity prediction from medical images has largely focused on learning representations from single-timepoint or snapshot scans. It has been shown that disease progression can be better characterized by temporal imaging. We therefore hypothesized that outcome predictions can be improved by utilizing the disease progression information from sequential images. We present a deep learning approach that leverages temporal progression information to improve clinical outcome predictions from single-timepoint images. In our method, a self-attention based Temporal Convolutional Netw…
Télécom SudParis, Institut Polytechnique de Paris（巴黎电信 SudParis，巴黎综合理工学院）；University College London（伦敦大学学院）；Digital Technologies and Artificial Intelligence Development Research Institute (AIRI)（数字技术与人工智能发展研究院（AIRI））
Do Pathology Foundation Models Encode Disease Progression? A Pseudotime Analysis of Visual Representations
Massachusetts Institute of Technology（麻省理工学院）；Department of Pathology, Dana-Farber Cancer Institute（达纳-法伯癌症研究所病理部）；Department of Data Science, Dana-Farber Cancer Institute（达纳-法伯癌症研究所数据科学部）；Brigham and Women's Hospital（布里奇沃特医院）；Harvard Medical School（哈佛医学院）
Alzheimers Disease Progression Prediction Based on Manifold Mapping of Irregularly Sampled Longitudinal Data
College of Computer Science and Technology, Huaqiao University（华侨大学计算机科学与技术学院）；Key Laboratory of Computer Vision and Machine Learning in Fujian Province（福建省计算机视觉与机器学习重点实验室）；College of Information Science and Engineering,Ritsumeikan University（立命馆大学信息科学与工程学院）
PIE: Simulating Disease Progression via Progressive Image Editing
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of California San Diego（加利福尼亚大学圣迭戈分校）；University of California, Berkeley（加利福尼亚大学伯克利分校）；University of Virginia（弗吉尼亚大学）；PediaMed AI；Northwestern University（西北大学）；Carnegie Mellon University（卡内基梅隆大学）
Flexible Multimodal Neuroimaging Fusion for Alzheimer's Disease Progression Prediction
Department of Computer Science and Engineering（计算机科学与工程系）；Department of Biomedical Informatics（生物医学信息学系）；Department of Neurology（神经病学系）；Translational Data Analytics Institute（转化数据分析研究所）
Medical University of Vienna（维也纳医科大学）；Centre for Eye Research Australia（澳大利亚眼科研究中心）；Royal Victorian Eye and Ear Hospital（皇家维多利亚眼耳医院）；The University of Melbourne（墨尔本大学）
Longitudinal imaging is capable of capturing the static ana\-to\-mi\-cal structures and the dynamic changes of the morphology resulting from aging or disease progression. Self-supervised learning allows to learn new representation from available large unlabelled data without any expert knowledge. We propose a deep learning self-supervised approach to model disease progression from longitudinal retinal optical coherence tomography (OCT). Our self-supervised model takes benefit from a generic time-related task, by learning to estimate the time interval between pairs of scans acquired from the sa…
Biomediq A/S（Biomediq公司）；University of Copenhagen（哥本哈根大学）；University College London（伦敦大学学院）
Disease progression modeling (DPM) using longitudinal data is a challenging task in machine learning for healthcare that can provide clinicians with better tools for diagnosis and monitoring of disease. Existing DPM algorithms neglect temporal dependencies among measurements and make parametric assumptions about biomarker trajectories. In addition, they do not model multiple biomarkers jointly and need to align subjects' trajectories. In this paper, recurrent neural networks (RNNs) are utilized to address these issues. However, in many cases, longitudinal cohorts contain incomplete data, which…
Robust forecasting of the future anatomical changes inflicted by an ongoing disease is an extremely challenging task that is out of grasp even for experienced healthcare professionals. Such a capability, however, is of great importance since it can improve patient management by providing information on the speed of disease progression already at the admission stage, or it can enrich the clinical trials with fast progressors and avoid the need for control arms by the means of digital twins. In this work, we develop a deep learning method that models the evolution of age-related disease by proce…
Graph Theory and GNNs to Unravel the Topographical Organization of Brain Lesions in Variants of Alzheimer's Disease Progression
Sorbonne Université（索邦大学）；Paris Brain Institute - ICM（巴黎脑研究所 - ICM）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；Inserm（法国国家健康与医学研究院）；AP-HP（巴黎公共医疗集团）；Hôpital de la Pitié Salpêtrière（皮蒂-萨尔皮特里耶医院）
Biomediq A/S（Biomediq公司）；Cerebriu A/S（Cerebriu公司）；University of Copenhagen（哥本哈根大学）；University College London（伦敦大学学院）；King’s College London（伦敦国王学院）
Quantitative characterization of disease progression using longitudinal data can provide long-term predictions for the pathological stages of individuals. This work studies the robust modeling of Alzheimer's disease progression using parametric methods. The proposed method linearly maps the individual's age to a disease progression score (DPS) and jointly fits constrained generalized logistic functions to the longitudinal dynamics of biomarkers as functions of the DPS using M-estimation. Robustness of the estimates is quantified using bootstrapping via Monte Carlo resampling, and the estimated…
RWTH Aachen University（亚琛工业大学）；University Hospital RWTH Aachen（亚琛工业大学附属医院）；University of Leeds（利兹大学）；University Hospital Heidelberg（海德堡大学附属医院）；University Hospital Aachen（亚琛大学附属医院）；CheckupPoint GmbH（CheckupPoint有限公司）；Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学MEVIS研究所）
Disease-modifying management aims to prevent deterioration and progression of the disease, not just relieve symptoms. Unfortunately, the development of necessary therapies is often hampered by the failure to recognize the presymptomatic disease and limited understanding of disease development. We present a generic solution for this problem by a methodology that allows the prediction of progression risk and morphology in individuals using a latent extrapolation optimization approach. To this end, we combined a regularized generative adversarial network (GAN) and a latent nearest neighbor algori…
Stanford University（斯坦福大学）；Stanford School of Medicine（斯坦福医学院）；University of California San Diego（加州大学圣地亚哥分校）；SRI International（斯坦福国际研究院）
Many neurological diseases are characterized by gradual deterioration of brain structure and function. Large longitudinal MRI datasets have revealed such deterioration, in part, by applying machine and deep learning to predict diagnosis. A popular approach is to apply Convolutional Neural Networks (CNN) to extract informative features from each visit of the longitudinal MRI and then use those features to classify each visit via Recurrent Neural Networks (RNNs). Such modeling neglects the progressive nature of the disease, which may result in clinically implausible classifications across visits…
Recent contrastive learning methods achieved state-of-the-art in low label regimes. However, the training requires large batch sizes and heavy augmentations to create multiple views of an image. With non-contrastive methods, the negatives are implicitly incorporated in the loss, allowing different images and modalities as pairs. Although the meta-information (i.e., age, sex) in medical imaging is abundant, the annotations are noisy and prone to class imbalance. In this work, we exploited already existing temporal information (different visits from a patient) in a longitudinal optical coherence…
ImageFlowNet: Forecasting Multiscale Image-Level Trajectories of Disease Progression with Irregularly-Sampled Longitudinal Medical Images
Yale University（耶鲁大学）；University of California, San Francisco（加州大学旧金山分校）；Mila - Quebec AI Institute（魁北克AI研究所）；Université de Montréal（蒙特利尔大学）；McGill University（麦吉尔大学）；Northern California Retina Vitreous Associates（北加州视网膜与玻璃体协会）
Introduction: It is challenging at baseline to predict when and which individuals who meet criteria for mild cognitive impairment (MCI) will ultimately progress to Alzheimer's disease (AD) dementia. Methods: A deep learning method is developed and validated based on MRI scans of 2146 subjects (803 for training and 1343 for validation) to predict MCI subjects' progression to AD dementia in a time-to-event analysis setting. Results: The deep learning time-to-event model predicted individual subjects' progression to AD dementia with a concordance index (C-index) of 0.762 on 439 ADNI testing MCI s…
Kyung Hee University（庆熙大学）；Sharif University of Technology（谢里夫理工大学）；Technical University of Munich（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Chest computed tomography (CT) has played an essential diagnostic role in assessing patients with COVID-19 by showing disease-specific image features such as ground-glass opacity and consolidation. Image segmentation methods have proven to help quantify the disease burden and even help predict the outcome. The availability of longitudinal CT series may also result in an efficient and effective method to reliably assess the progression of COVID-19, monitor the healing process and the response to different therapeutic strategies. In this paper, we propose a new framework to identify infection at…
Cornell（康奈尔大学）；Electrical and Computer Eng, Cornell（康奈尔大学电气与计算机工程学院）
In this work, we consider the problem of predicting the course of a progressive disease, such as cancer or Alzheimer's. Progressive diseases often start with mild symptoms that might precede a diagnosis, and each patient follows their own trajectory. Patient trajectories exhibit wild variability, which can be associated with many factors such as genotype, age, or sex. An additional layer of complexity is that, in real life, the amount and type of data available for each patient can differ significantly. For example, for one patient we might have no prior history, whereas for another patient we…
University College London（伦敦大学学院）；Inria（法国国家信息与自动化研究所）；Université Côte d'Azur（蔚蓝海岸大学）
Here we present DIVE: Data-driven Inference of Vertexwise Evolution. DIVE is an image-based disease progression model with single-vertex resolution, designed to reconstruct long-term patterns of brain pathology from short-term longitudinal data sets. DIVE clusters vertex-wise biomarker measurements on the cortical surface that have similar temporal dynamics across a patient population, and concurrently estimates an average trajectory of vertex measurements in each cluster. DIVE uniquely outputs a parcellation of the cortex into areas with common progression patterns, leading to a new signature…
Endoscopy is a routine imaging technique used for both diagnosis and minimally invasive surgical treatment. While the endoscopy video contains a wealth of information, tools to capture this information for the purpose of clinical reporting are rather poor. In date, endoscopists do not have any access to tools that enable them to browse the video data in an efficient and user friendly manner. Fast and reliable video retrieval methods could for example, allow them to review data from previous exams and therefore improve their ability to monitor disease progression. Deep learning provides new ave…
Alzheimer's disease (AD) is known as one of the major causes of dementia and is characterized by slow progression over several years, with no treatments or available medicines. In this regard, there have been efforts to identify the risk of developing AD in its earliest time. While many of the previous works considered cross-sectional analysis, more recent studies have focused on the diagnosis and prognosis of AD with longitudinal or time series data in a way of disease progression modeling (DPM). Under the same problem settings, in this work, we propose a novel computational framework that ca…
Scientific Computing and Imaging Institute, University of Utah（犹他大学科学计算与成像研究所）；Department of Radiology and Imaging Sciences, University of Utah（犹他大学放射学与影像科学系）
Knowledge of what spatial elements of medical images deep learning methods use as evidence is important for model interpretability, trustiness, and validation. There is a lack of such techniques for models in regression tasks. We propose a method, called visualization for regression with a generative adversarial network (VR-GAN), for formulating adversarial training specifically for datasets containing regression target values characterizing disease severity. We use a conditional generative adversarial network where the generator attempts to learn to shift the output of a regressor through cre…
