FB-Diff: Fourier Basis-guided Diffusion for Temporal Interpolation of 4D Medical Imaging
Computer Aided Medical Procedures, Technical University of Munich（技术大学慕尼黑计算机辅助医学程序）；Institute of Medical Robotics, Shanghai Jiao Tong University（上海交通大学医学机器人研究所）；Munich Center for Machine Learning, Munich（慕尼黑机器学习中心）；School of Computing, Macquarie University（麦考瑞大学计算机学院）
The temporal interpolation task for 4D medical imaging, plays a crucial role in clinical practice of respiratory motion modeling. Following the simplified linear-motion hypothesis, existing approaches adopt optical flow-based models to interpolate intermediate frames. However, realistic respiratory motions should be nonlinear and quasi-periodic with specific frequencies. Intuited by this property, we resolve the temporal interpolation task from the frequency perspective, and propose a Fourier basis-guided Diffusion model, termed FB-Diff. Specifically, due to the regular motion discipline of respiration, physiological motion priors are introduced to describe general characteristics of temporal data distributions. Then a Fourier motion operator is elaborately devised to extract Fourier bases by incorporating physiological motion priors and case-specific spectral information in the feature space of Variational Autoencoder. Well-learned Fourier bases can better simulate respiratory motions with motion patterns of specific frequencies. Conditioned on starting and ending frames, the diffusion model further leverages well-learned Fourier bases via the basis interaction operator, which promotes the temporal interpolation task in a generative manner. Extensive results demonstrate that FB-Diff achieves state-of-the-art (SOTA) perceptual performance with better temporal consistency while maintaining promising reconstruction metrics. Codes are available.
Shanghai Jiao Tong University（上海交通大学）；University of Sydney（悉尼大学）
Dynamic medical imaging is usually limited in application due to the large radiation doses and longer image scanning and reconstruction times. Existing methods attempt to reduce the dynamic sequence by interpolating the volumes between the acquired image volumes. However, these methods are limited to either 2D images and/or are unable to support large variations in the motion between the image volume sequences. In this paper, we present a spatiotemporal volumetric interpolation network (SVIN) designed for 4D dynamic medical images. SVIN introduces dual networks: first is the spatiotemporal mot…
Digital medical images are always displayed scaled to fit particular view. Interpolation is responsible for this scaling, and if not done properly, can significantly degrade diagnostic image quality. However, theoretically-optimal interpolation algorithms may also be the most time-consuming and impractical. We propose a new approach, adapted to the needs of digital medical imaging, to combine high interpolation speed and superior L2-optimal image quality.
South China University of Technology（华南理工大学）；Rochester Institute of Technology（罗切斯特理工学院）
We introduce a novel frame-interpolation-based method for slice imputation to improve segmentation accuracy for anisotropic 3D medical images, in which the number of slices and their corresponding segmentation labels can be increased between two consecutive slices in anisotropic 3D medical volumes. Unlike previous inter-slice imputation methods, which only focus on the smoothness in the axial direction, this study aims to improve the smoothness of the interpolated 3D medical volumes in all three directions: axial, sagittal, and coronal. The proposed multitask inter-slice imputation method, in …
Slice interpolation is a fast growing field in medical image processing. Intensity-based interpolation and object-based interpolation are two major groups of methods in the literature. In this paper, we describe an object-oriented, optimization method based on a modified version of curvature-based image registration, in which a displacement field is computed for the missing slice between two known slices and used to interpolate the intensities of the missing slice. The proposed approach is evaluated quantitatively by using the Mean Squared Difference (MSD) as a metric. The produced results als…
The scarcity of pixel-level annotation is a prevalent problem in medical image segmentation tasks. In this paper, we introduce a novel regularization strategy involving interpolation-based mixing for semi-supervised medical image segmentation. The proposed method is a new consistency regularization strategy that encourages segmentation of interpolation of two unlabelled data to be consistent with the interpolation of segmentation maps of those data. This method represents a specific type of data-adaptive regularization paradigm which aids to minimize the overfitting of labelled data under high…
Phase-Aligned Finite-Fourier Periodic Deformation for 4D Medical Image Interpolation
Southern University of Science and Technology（南方科技大学）；Shenzhen University of Advanced Technology（深圳理工大学）；Shenzhen University Medical School（深圳大学医学院）；Shenzhen University（深圳大学）；Beihang University（北京航空航天大学）
Low-Rank Velocity Fields as a Structural Prior for Unsupervised 4D Medical Image Interpolation
Southern University of Science and Technology（南方科技大学）；Shenzhen University of Advanced Technology（深圳先进技术研究院）；Shenzhen University Medical School（深圳大学医学院）；Shenzhen University（深圳大学）；Research Institute of Trustworthy Autonomous Systems（可信自主系统研究院）
University of Padova（帕多瓦大学）；Padua University Hospital（帕多瓦大学医院）；University of Genova（热那亚大学）
To analyse multimodal 3-dimensional medical images, interpolation is required for resampling which - unavoidably - introduces an interpolation error. In this work we consider three segmented 3-dimensional images resampled with three different neuroimaging software tools for comparing undersampling and oversampling strategies and to identify where the oversampling error lies. The results indicate that undersampling to the lowest image size is advantageous in terms of mean value per segment errors and that the oversampling error is larger where the gradient is steeper, showing a Gibbs effect.
Despite the surge of deep learning in the past decade, some users are skeptical to deploy these models in practice due to their black-box nature. Specifically, in the medical space where there are severe potential repercussions, we need to develop methods to gain confidence in the models' decisions. To this end, we propose a novel medical imaging generative adversarial framework, medXGAN (medical eXplanation GAN), to visually explain what a medical classifier focuses on in its binary predictions. By encoding domain knowledge of medical images, we are able to disentangle anatomical structure an…
Manifold learning has been successfully applied to a variety of medical imaging problems. Its use in real-time applications requires fast projection onto the low-dimensional space. To this end, out-of-sample extensions are applied by constructing an interpolation function that maps from the input space to the low-dimensional manifold. Commonly used approaches such as the Nyström extension and kernel ridge regression require using all training points. We propose an interpolation function that only depends on a small subset of the input training data. Consequently, in the testing phase each new …
MambaMIM: Pre-training Mamba with State Space Token Interpolation and its Application to Medical Image Segmentation
School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China（科学技术大学生物医学工程学院）；Suzhou Institute for Advanced Research, University of Science and Technology of China（科学技术大学苏州市先进研究所）；Center for Medical Imaging, Robotics, and Analytic Computing & LEarning (MIRACLE), Suzhou Institute for Advanced Research, USTC（医学影像、机器人与分析计算与学习中心）；Department of Automation, Shanghai Jiao Tong University（上海交通大学自动化系）；Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University（上海交通大学图像处理与模式识别研究所）；Institute of Medical Robotics, Shanghai Jiao Tong University（上海交通大学医学机器人研究所）；State Key Laboratory of Precision and Intelligent Chemistry, University of Science and Technology of China（科学技术大学精密与智能化学国家重点实验室）
Continuous Filtered Backprojection by Learnable Interpolation Network
School of Mathematics and Statistics, Xi’an Jiaotong University（数学与统计学学院，西安交通大学）；School of Biomedical Engineering, Southern Medical University（生物医学工程学院，南方医科大学）；School of Life Science and Technology, Xi’an Jiaotong University（生命科学与技术学院，西安交通大学）
We propose to learn a probabilistic motion model from a sequence of images. Besides spatio-temporal registration, our method offers to predict motion from a limited number of frames, useful for temporal super-resolution. The model is based on a probabilistic latent space and a novel temporal dropout training scheme. This enables simulation and interpolation of realistic motion patterns given only one or any subset of frames of a sequence. The encoded motion also allows to be transported from one subject to another without the need of inter-subject registration. An unsupervised generative defor…
South China University of Technology（华南理工大学）；University of Utah（犹他大学）
We introduce the idea of inter-slice image augmentation whereby the numbers of the medical images and the corresponding segmentation labels are increased between two consecutive images in order to boost medical image segmentation accuracy. Unlike conventional data augmentation methods in medical imaging, which only increase the number of training samples directly by adding new virtual samples using simple parameterized transformations such as rotation, flipping, scaling, etc., we aim to augment data based on the relationship between two consecutive images, which increases not only the number b…
In this paper a fast triangular mesh based registration method is proposed. Having Template and Reference images as inputs, the template image is triangulated using a content adaptive mesh generation algorithm. Considering the pixel values at mesh nodes, interpolated using spline interpolation method for both of the images, the energy functional needed for image registration is minimized. The minimization process was achieved using a mesh based discretization of the distance measure and regularization term which resulted in a sparse system of linear equations, which due to the smaller size in …
MoSt-DSA: Modeling Motion and Structural Interactions for Direct Multi-Frame Interpolation in DSA Images
School of EIC, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）；Union Hospital, Tongji Medical College, Huazhong University of Science and Technology（华中科技大学同济医学院附属协和医院）
Amsterdam University Medical Centers（阿姆斯特丹大学医学中心）；University of Amsterdam（阿姆斯特丹大学）；Amsterdam Cardiovascular Sciences（阿姆斯特丹心血管科学研究所）；Informatics Institute, University of Amsterdam（阿姆斯特丹大学信息学研究所）
High-resolution medical images are beneficial for analysis but their acquisition may not always be feasible. Alternatively, high-resolution images can be created from low-resolution acquisitions using conventional upsampling methods, but such methods cannot exploit high-level contextual information contained in the images. Recently, better performing deep-learning based super-resolution methods have been introduced. However, these methods are limited by their supervised character, i.e. they require high-resolution examples for training. Instead, we propose an unsupervised deep learning semanti…
Technical University of Munich（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；ETH Zurich（苏黎世联邦理工学院）
Medical image segmentation is one of the major challenges addressed by machine learning methods. Yet, deep learning methods profoundly depend on a large amount of annotated data, which is time-consuming and costly. Though, semi-supervised learning methods approach this problem by leveraging an abundant amount of unlabeled data along with a small amount of labeled data in the training process. Recently, MixUp regularizer has been successfully introduced to semi-supervised learning methods showing superior performance. MixUp augments the model with new data points through linear interpolation of…
Medical image slice interpolation is an active field of research. The methods for this task can be categorized into two broad groups: intensity-based and object-based interpolation methods. While intensity-based methods are generally easier to perform and less computationally expensive, object-based methods are capable of producing more accurate results and account for deformable changes in the objects within the slices. In this paper, performance of two well-known object-based interpolation methods is analyzed and compared. Here, a deformable registration-based method specifically designed fo…
End-to-End Learning of Multi-Organ Implicit Surfaces from 3D Medical Imaging Data
University of Strasbourg, CNRS, INSERM, ICube, UMR7357, Strasbourg, France Institute of Image-Guided Surgery, IHU Strasbourg, Strasbourg, France Department of Diagnostic Radiology, McGill University, Montreal, Canada Augmented Intelligence \& Precision Health Laboratory (AIPHL), McGill University Health Centre Research Institute, Montreal, Canada
Deep learning-based single image super-resolution (SISR) methods face various challenges when applied to 3D medical volumetric data (i.e., CT and MR images) due to the high memory cost and anisotropic resolution, which adversely affect their performance. Furthermore, mainstream SISR methods are designed to work over specific upsampling factors, which makes them ineffective in clinical practice. In this paper, we introduce a Spatially Aware Interpolation NeTwork (SAINT) for medical slice synthesis to alleviate the memory constraint that volumetric data poses. Compared to other super-resolution …
Latent Diffusion Autoencoders: Toward Efficient and Meaningful Unsupervised Representation Learning in Medical Imaging
Department of Electrical and Information Engineering (DIEI), University of Cassino and Southern Lazio（电气与信息工程系（DIEI），卡斯诺和南部拉齐亚大学）；Diagnostic Image Analysis Group, Radboud University Medical Center（影像诊断分析组，拉德堡德大学医学中心）；Department of Medical Imaging, Radboud University Medical Center（医学成像系，拉德堡德大学医学中心）
Background and Objective. B-spline interpolation (BSI) is a popular technique in the context of medical imaging due to its adaptability and robustness in 3D object modeling. A field that utilizes BSI is Image Guided Surgery (IGS). IGS provides navigation using medical images, which can be segmented and reconstructed into 3D models, often through BSI. Image registration tasks also use BSI to align pre-operative data to intra-operative data. However, such IGS tasks are computationally demanding, especially when applied to 3D medical images, due to the complexity and amount of data involved. Ther…
Counterfactual Explanations for Medical Image Classification and Regression using Diffusion Autoencoder
Technical University of Munich（慕尼黑工业大学）；Klinikum rechts der Isar（右侧伊萨尔医院）；Institute for Artificial Intelligence and Computer Science in Medicine, Technical University of Munich（慕尼黑工业大学人工智能与医学计算机科学研究所）；Computer Aided Medical Procedures, Technical University of Munich（慕尼黑工业大学计算机辅助医疗程序研究所）；AI for Image-Guided Diagnosis and Therapy, Technical University of Munich（慕尼黑工业大学图像引导诊断与治疗人工智能研究所）
MCICSAM: Monte Carlo-guided Interpolation Consistency Segment Anything Model for Semi-Supervised Prostate Zone Segmentation
College of Medicine and Biological Information Engineering, Northeastern University（东北大学医学与生物信息工程学院）；Shengjing Hospital of China Medical University（中国医科大学附属盛京医院）；University of Chicago（芝加哥大学）；University of Dundee（邓迪大学）；School of Health Management, China Medical University（中国医科大学健康管理学院）
Informed Sinogram Interpolation for Sparse View Reconstruction
Technische Universität München（慕尼黑工业大学）；Waygate Technologies, Baker Hughes Digital Solutions GmbH（韦盖特技术，贝克休斯数字解决方案有限公司）；Technische Universität Darmstadt（达姆施塔特工业大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
Temporal interpolation has the potential to be a powerful tool for video compression. Existing methods for frame interpolation do not discriminate between video textures and generally invoke a single general model capable of interpolating a wide range of video content. However, past work on video texture analysis and synthesis has shown that different textures exhibit vastly different motion characteristics and they can be divided into three classes (static, dynamic continuous and dynamic discrete). In this work, we study the impact of video textures on video frame interpolation, and propose a…
Recent years have witnessed the emergence of 3D medical imaging techniques with the development of 3D sensors and technology. Due to the presence of noise in image acquisition, registration researchers focused on an alternative way to represent medical images. An alternative way to analyze medical imaging is by understanding the 3D shapes represented in terms of point-cloud. Though in the medical imaging community, 3D point-cloud processing is not a ``go-to'' choice, it is a ``natural'' way to capture 3D shapes. However, as the number of samples for medical images are small, researchers have u…
