In this paper, we propose a predictive regression model for longitudinal images with missing data based on large deformation diffeomorphic metric mapping (LDDMM) and deep neural networks. Instead of directly predicting image scans, our model predicts a vector momentum sequence associated with a baseline image. This momentum sequence parameterizes the original image sequence in the LDDMM framework and lies in the tangent space of the baseline image, which is Euclidean. A recurrent network with long term-short memory (LSTM) units encodes the time-varying changes in the vector-momentum sequence, and a convolutional neural network (CNN) encodes the baseline image of the vector momenta. Features extracted by the LSTM and CNN are fed into a decoder network to reconstruct the vector momentum sequence, which is used for the image sequence prediction by deforming the baseline image with LDDMM shooting. To handle the missing images at some time points, we adopt a binary mask to ignore their reconstructions in the loss calculation. We evaluate our model on synthetically generated images and the brain MRIs from the OASIS dataset. Experimental results demonstrate the promising predictions of the spatiotemporal changes in both datasets, irrespective of large or subtle changes in longitudinal image sequences.
JPGNet: Joint Predictive Filtering and Generative Network for Image Inpainting
College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);University of South Carolina(南卡罗来纳大学);Alibaba Group(阿里巴巴集团);Cleveland State University(克利夫兰州立大学);Nanyang Technological University(南洋理工大学);Zhejiang Sci-Tech University(浙江理工大学)
LoCI-DiffCom: Longitudinal Consistency-Informed Diffusion Model for 3D Infant Brain Image Completion
School of Biomedical Engineering, ShanghaiTech University(上海科技大学生物医学工程学院);Shanghai United Imaging Intelligence Co., Ltd(上海联影智能医疗科技有限公司);Shanghai Clinical Research and Trail Center(上海临床研究中心)
Longitudinal Prediction of Postnatal Brain Magnetic Resonance Images via a Metamorphic Generative Adversarial Network
Nanjing University of Information Science and Technology(南京信息工程大学);Radboud University Medical Center(拉德堡德大学医学中心);Shandong University (Weihai)(山东大学(威海))
Multi-View Imputation and Cross-Attention Network Based on Incomplete Longitudinal and Multimodal Data for Conversion Prediction of Mild Cognitive Impairment
Missing Data as Augmentation in the Earth Observation Domain: A Multi-View Learning Approach
Department of Computer Science, University of Kaiserslautern-Landau (RPTU)(卡斯尔拉特-兰道大学计算机科学系);SDS, German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心(DFKI))