Neural Radiance Fields and 3D Gaussian Splatting have revolutionized 3D reconstruction and novel-view synthesis task. However, achieving photorealistic rendering from extreme novel viewpoints remains challenging, as artifacts persist across representations. In this work, we introduce Difix3D+, a novel pipeline designed to enhance 3D reconstruction and novel-view synthesis through single-step diffusion models. At the core of our approach is Difix, a single-step image diffusion model trained to enhance and remove artifacts in rendered novel views caused by underconstrained regions of the 3D representation. Difix serves two critical roles in our pipeline. First, it is used during the reconstruction phase to clean up pseudo-training views that are rendered from the reconstruction and then distilled back into 3D. This greatly enhances underconstrained regions and improves the overall 3D representation quality. More importantly, Difix also acts as a neural enhancer during inference, effectively removing residual artifacts arising from imperfect 3D supervision and the limited capacity of current reconstruction models. Difix3D+ is a general solution, a single model compatible with both NeRF and 3DGS representations, and it achieves an average 2$\times$ improvement in FID score over baselines while maintaining 3D consistency.
We propose a novel 3d colored shape reconstruction method from a single RGB image through diffusion model. Diffusion models have shown great development potentials for high-quality 3D shape generation. However, most existing work based on diffusion models only focus on geometric shape generation, they cannot either accomplish 3D reconstruction from a single image, or produce 3D geometric shape with color information. In this work, we propose to reconstruct a 3D colored shape from a single RGB image through a novel conditional diffusion model. The reverse process of the proposed diffusion model…
Improving Neural Indoor Surface Reconstruction with Mask-Guided Adaptive Consistency Constraints
College of Information Engineering, Zhejiang University of Technology（浙江工业大学信息工程学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）
eBIRD: Event-based Intensity Image Reconstruction Using Controllable Diffusion Models
Institute of Engineering Sciences, Universidad de O’Higgins（奥伊金斯大学工程科学学院）；The Iniciativa de Datos e Inteligencia Artificial, University of Chile（智利大学数据与人工智能倡议）
Product-of-Gaussian-Mixture Diffusion Models for Joint Nonlinear MRI Reconstruction
Graz University of Technology（格拉茨技术大学）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；Biomedical Imaging Group and Center for Biomedical Imaging（生物医学成像组和生物医学成像中心）
Iterative Diffusion-Refined Neural Attenuation Fields for Multi-Source Stationary CT Reconstruction: NAF Meets Diffusion Model
School of Information Engineering, Nanchang University（南昌大学信息工程学院）；School of Mathematics and Computer Sciences, Nanchang University（南昌大学数学与计算机科学学院）；School of Biomedical Engineering, Sun Yat-sen University（中山大学生物医学工程学院）；Laboratory of Image Science and Technology, School of Computer Science and Engineering, and the Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Ministry of Education, Southeast University（东南大学计算机科学与工程学院图像科学与技术实验室，以及教育部新一代人工智能技术及其交叉应用重点实验室）
We propose DiffuStereo, a novel system using only sparse cameras (8 in this work) for high-quality 3D human reconstruction. At its core is a novel diffusion-based stereo module, which introduces diffusion models, a type of powerful generative models, into the iterative stereo matching network. To this end, we design a new diffusion kernel and additional stereo constraints to facilitate stereo matching and depth estimation in the network. We further present a multi-level stereo network architecture to handle high-resolution (up to 4k) inputs without requiring unaffordable memory footprint. Give…
Diffusion-Encoding Gaussian Field for Joint k-q dMRI Reconstruction
School of Information Engineering, Nanchang University（南昌大学信息工程学院）；Academy of Medical Engineering and Translational Medicine, Medical School, Faculty of Medicine, Tianjin University（天津大学医学院医学工程与转化医学研究院）；Research Center for Medical AI, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院医学人工智能研究中心）
The Australian National University（澳大利亚国立大学）；University College London（伦敦大学学院）
This paper proposes a multi-shell sampling scheme and corresponding transforms for the accurate reconstruction of the diffusion signal in diffusion MRI by expansion in the spherical polar Fourier (SPF) basis. The sampling scheme uses an optimal number of samples, equal to the degrees of freedom of the band-limited diffusion signal in the SPF domain, and allows for computationally efficient reconstruction. We use synthetic data sets to demonstrate that the proposed scheme allows for greater reconstruction accuracy of the diffusion signal than the multi-shell sampling schemes obtained using the …
Peking University（北京大学）；Academy for Advanced Interdisciplinary Studies（前沿交叉学科研究院）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
We propose a novel and unified method, measurement-conditioned denoising diffusion probabilistic model (MC-DDPM), for under-sampled medical image reconstruction based on DDPM. Different from previous works, MC-DDPM is defined in measurement domain (e.g. k-space in MRI reconstruction) and conditioned on under-sampling mask. We apply this method to accelerate MRI reconstruction and the experimental results show excellent performance, outperforming full supervision baseline and the state-of-the-art score-based reconstruction method. Due to its generative nature, MC-DDPM can also quantify the unce…
ETH Zurich（苏黎世联邦理工学院）；Paul Scherrer Institute（保罗·谢勒研究所）
This work introduces and characterizes a fast parameterless filter based on the Helgason-Ludwig consistency conditions, used to improve the accuracy of analytical reconstructions of tomographic undersampled datasets. The filter, acting in the Radon domain, extrapolates intermediate projections between those existing. The resulting sinogram, doubled in views, is then reconstructed by a standard analytical method. Experiments with simulated data prove that the peak-signal-to-noise ratio of the results computed by filtered backprojection is improved up to 5-6 dB, if the filter is used prior to re…
End-to-end trainable models have reached the performance of traditional handcrafted compression techniques on videos and images. Since the parameters of these models are learned over large training sets, they are not optimal for any given image to be compressed. In this paper, we propose an instance-based fine-tuning of a subset of decoder's bias to improve the reconstruction quality in exchange for extra encoding time and minor additional signaling cost. The proposed method is applicable to any end-to-end compression methods, improving the state-of-the-art neural image compression BD-rate by …
Steerable Conditional Diffusion for Out-of-Distribution Adaptation in Medical Image Reconstruction
University College London（伦敦大学学院）；Atinary Technologies（Atinary科技公司）；University of Bremen（不来梅大学）；Korea Advanced Institute of Science and Technology（韩国科学技术院）；Ajou University School of Medicine（亚洲大学医学院）；The Chinese University of Hong Kong（香港中文大学）
Diffusion models are a powerful class of generative models that iteratively denoise samples to produce data. While many works have focused on the number of iterations in this sampling procedure, few have focused on the cost of each iteration. We find that adding a simple ViT-style patching transformation can considerably reduce a diffusion model's sampling time and memory usage. We justify our approach both through an analysis of the diffusion model objective, and through empirical experiments on LSUN Church, ImageNet 256, and FFHQ 1024. We provide implementations in Tensorflow and Pytorch.
DVG-Diffusion: Dual-View Guided Diffusion Model for CT Reconstruction from X-Rays
Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；The University of Hong Kong（香港大学）
