Diffusion models are highly expressive image priors for Bayesian inverse problems. However, most diffusion models cannot operate on large-scale, high-dimensional data due to high training and inference costs. In this work, we introduce a Plug-and-play algorithm for 3D stochastic inference with latent diffusion prior (PSI3D) to address massive ($1024\times 1024\times 128$) volumes. Specifically, we formulate a Markov chain Monte Carlo approach to reconstruct each two-dimensional (2D) slice by sampling from a 2D latent diffusion model. To enhance inter-slice consistency, we also incorporate total variation (TV) regularization stochastically along the concatenation axis. We evaluate our performance on optical coherence tomography (OCT) super-resolution. Our method significantly improves reconstruction quality for large-scale scientific imaging compared to traditional and learning-based baselines, while providing robust and credible reconstructions.
Romanian Institute of Science and Technology（罗马尼亚科学技术研究所）；Babeş-Bolyai University（巴贝什-博尧依大学）
Anomaly detection for Magnetic Resonance Images (MRIs) can be solved with unsupervised methods by learning the distribution of healthy images and identifying anomalies as outliers. In presence of an additional dataset of unlabelled data containing also anomalies, the task can be framed as a semi-supervised task with negative and unlabelled sample points. Recently, in Albu et al., 2020, we have proposed a slice-wise semi-supervised method for tumour detection based on the computation of a dissimilarity function in the latent space of a Variational AutoEncoder, trained on unlabelled data. The di…
Technical University of Darmstadt（达姆施塔特工业大学）；Universität der Bundeswehr München（慕尼黑联邦国防军大学）；Centre for Cognitive Science, TU Darmstadt（达姆施塔特工业大学认知科学中心）；Hessian Center for AI (hessian.AI)（黑森人工智能中心（hessian.AI））
Model inversion attacks (MIAs) aim to create synthetic images that reflect the class-wise characteristics from a target classifier's private training data by exploiting the model's learned knowledge. Previous research has developed generative MIAs that use generative adversarial networks (GANs) as image priors tailored to a specific target model. This makes the attacks time- and resource-consuming, inflexible, and susceptible to distributional shifts between datasets. To overcome these drawbacks, we present Plug & Play Attacks, which relax the dependency between the target model and image prio…
Comparative Analysis of GAN and Diffusion for MRI-to-CT translation
Department of Computer Science, University of Copenhagen, Copenhagen, Denmark（哥本哈根大学计算机科学系）；Department of Oncology, Rigshospitalet, Copenhagen, Denmark（哥本哈根医院肿瘤科）
Heidelberg University（海德堡大学）；University Medical Center Mannheim（曼海姆大学医学中心）
At present, lesion segmentation is still performed manually (or semi-automatically) by medical experts. To facilitate this process, we contribute a fully-automatic lesion segmentation pipeline. This work proposes a method as a part of the LiTS (Liver Tumor Segmentation Challenge) competition for ISBI 17 and MICCAI 17 comparing methods for automatics egmentation of liver lesions in CT scans. By utilizing cascaded, densely connected 2D U-Nets and a Tversky-coefficient based loss function, our framework achieves very good shape extractions with high detection sensitivity, with competitive scores …
Structural magnetic resonance imaging (MRI) has been widely utilized for analysis and diagnosis of brain diseases. Automatic segmentation of brain tumors is a challenging task for computer-aided diagnosis due to low-tissue contrast in the tumor subregions. To overcome this, we devise a novel pixel-wise segmentation framework through a convolutional 3D to 2D MR patch conversion model to predict class labels of the central pixel in the input sliding patches. Precisely, we first extract 3D patches from each modality to calibrate slices through the squeeze and excitation (SE) block. Then, the outp…
Plug-and-Play optimization recently emerged as a powerful technique for solving inverse problems by plugging a denoiser into a classical optimization algorithm. The denoiser accounts for the regularization and therefore implicitly determines the prior knowledge on the data, hence replacing typical handcrafted priors. In this paper, we extend the concept of plug-and-play optimization to use denoisers that can be parameterized for non-constant noise variance. In that aim, we introduce a preconditioning of the ADMM algorithm, which mathematically justifies the use of such an adjustable denoiser. …
With the fast growth of immersive video sequences, achieving seamless and high-quality compressed 3D content is even more critical. MPEG recently developed a video-based point cloud compression (V-PCC) standard for dynamic point cloud coding. However, reconstructed point clouds using V-PCC suffer from different artifacts, including losing data during pre-processing before applying existing video coding techniques, e.g., High-Efficiency Video Coding (HEVC). Patch generations and self-occluded points in the 3D to the 2D projection are the main reasons for missing data using V-PCC. This paper pro…
School of Electrical and Computer Engineering, Purdue University（普渡大学电气与计算机工程学院）；Eli Lilly and Company（礼来公司）；Department of Mathematics, Purdue University（普渡大学数学系）
Inverse problems spanning four or more dimensions such as space, time and other independent parameters have become increasingly important. State-of-the-art 4D reconstruction methods use model based iterative reconstruction (MBIR), but depend critically on the quality of the prior modeling. Recently, plug-and-play (PnP) methods have been shown to be an effective way to incorporate advanced prior models using state-of-the-art denoising algorithms. However, state-of-the-art denoisers such as BM4D and deep convolutional neural networks (CNNs) are primarily available for 2D or 3D images and extendi…
Institute for Artificial Intelligence（人工智能研究所）；University of Georgia（佐治亚大学）；Shanghai Jiao Tong University（上海交通大学）
Segmenting an entire 3D image often has high computational complexity and requires large memory consumption; by contrast, performing volumetric segmentation in a slice-by-slice manner is efficient but does not fully leverage the 3D data. To address this challenge, we propose a multi-dimensional attention network (MDA-Net) to efficiently integrate slice-wise, spatial, and channel-wise attention into a U-Net based network, which results in high segmentation accuracy with a low computational cost. We evaluate our model on the MICCAI iSeg and IBSR datasets, and the experimental results demonstrate…
PURe: A Plug-and-Play Product-Unit Residual Module for Vision Networks
Department of Mathematics, Informatics and Technology, University of Applied Sciences Koblenz（科隆应用科学大学数学、信息学与技术系）；Technical University of Munich（慕尼黑技术大学）
Purdue University（普渡大学）；Eli Lilly and Company（礼来公司）
There is an increasing need to reconstruct objects in four or more dimensions corresponding to space, time and other independent parameters. The best 4D reconstruction algorithms use regularized iterative reconstruction approaches such as model based iterative reconstruction (MBIR), which depends critically on the quality of the prior modeling. Recently, Plug-and-Play methods have been shown to be an effective way to incorporate advanced prior models using state-of-the-art denoising algorithms designed to remove additive white Gaussian noise (AWGN). However, state-of-the-art denoising algorith…
University of Sydney（悉尼大学）；Sydney Neuroimaging Analysis Centre（悉尼神经影像分析中心）
Diffusion Weighted Imaging (DWI) is an advanced imaging technique commonly used in neuroscience and neurological clinical research through a Diffusion Tensor Imaging (DTI) model. Volumetric scalar metrics including fractional anisotropy, mean diffusivity, and axial diffusivity can be derived from the DTI model to summarise water diffusivity and other quantitative microstructural information for clinical studies. However, clinical practice constraints can lead to sub-optimal DWI acquisitions with missing slices (either due to a limited field of view or the acquisition of disrupted slices). To a…
College of Health Science and Environmental Engineering, Shenzhen Technology University（深圳科技大学健康科学与环境工程学院）；University Medical Center Göttingen（哥廷根大学医学中心）；Guangdong-Hongkong-Macau CNS Regeneration Institute, Key Laboratory of CNS Regeneration (Jinan University)-Ministry of Education, Jinan University（广东-港澳-澳门神经科学再生研究所，神经科学再生重点实验室（暨南大学）-教育部，暨南大学）；Department of Radiology, The First Hospital of Tsinghua University（清华大学第一医院放射科）；Center for Biomedical Imaging Research, Department of Biomedical Engineering, School of Medicine, Tsinghua University（清华大学生物医学成像研究中心，生物医学工程系，医学院）；Key Laboratory for Biomedical Engineering of Ministry of Education, College of Biomedical Engineering and Instrument Science, Zhejiang University（教育部生物医学工程重点实验室，生物医学工程与仪器科学学院，浙江大学）；Department of Endocrinology and Metabolism, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine（浙江大学医学院邵逸夫医院内分泌科）；Lab of Molecular Imaging and Medical Intelligence, Department of Radiology, Longgang Central Hospital of Shenzhen (Shenzhen Clinical Medical College, Guangzhou University of Chinese Medicine（分子成像与医学智能实验室，放射科，深圳龙岗中心医院（深圳临床医学院，广州中医药大学；龙岗临床学院，汕头大学医学院））；Longgang Clinical Institute of Shantou University Medical College)
This paper investigates the application of deep learning models for lung Computed Tomography (CT) image analysis. Traditional deep learning frameworks encounter compatibility issues due to variations in slice numbers and resolutions in CT images, which stem from the use of different machines. Commonly, individual slices are predicted and subsequently merged to obtain the final result; however, this approach lacks slice-wise feature learning and consequently results in decreased performance. We propose a novel slice selection method for each CT dataset to address this limitation, effectively fi…
We demonstrate a motion-free intensity diffraction tomography technique that enables direct inversion of 3D phase and absorption from intensity-only measurements for weakly scattering samples. We derive a novel linear forward model, featuring slice-wise phase and absorption transfer functions using angled illumination. This new framework facilitates flexible and efficient data acquisition, enabling arbitrary sampling of the illumination angles. The reconstruction algorithm performs 3D synthetic aperture using a robust, computation and memory efficient slice-wise deconvolution to achieve resolu…
Generating Synthetic Contrast-Enhanced Chest CT Images from Non-Contrast Scans Using Slice-Consistent Brownian Bridge Diffusion Network
Department of Medical Imaging, University of Saskatchewan（医学成像系，萨斯喀彻温大学）；Department of Electrical and Computer Engineering, University of Saskatchewan（电气与计算机工程系，萨斯喀彻温大学）
