The ability of deep image prior (DIP) to recover high-quality images from incomplete or corrupted measurements has made it popular in inverse problems in image restoration and medical imaging including magnetic resonance imaging (MRI). However, conventional DIP suffers from severe overfitting and spectral bias effects. In this work, we first provide an analysis of how DIP recovers information from undersampled imaging measurements by analyzing the training dynamics of the underlying networks in the kernel regime for different architectures. This study sheds light on important underlying properties for DIP-based recovery. Current research suggests that incorporating a reference image as network input can enhance DIP's performance in image reconstruction compared to using random inputs. However, obtaining suitable reference images requires supervision, and raises practical difficulties. In an attempt to overcome this obstacle, we further introduce a self-driven reconstruction process that concurrently optimizes both the network weights and the input while eliminating the need for training data. Our method incorporates a novel denoiser regularization term which enables robust and stable joint estimation of both the network input and reconstructed image. We demonstrate that our self-guided method surpasses both the original DIP and modern supervised methods in terms of MR image reconstruction performance and outperforms previous DIP-based schemes for image inpainting.
Facebook AI Research（Facebook人工智能研究）；fastMRI team（fastMRI团队）
Deep learning approaches to accelerated MRI take a matrix of sampled Fourier-space lines as input and produce a spatial image as output. In this work we show that by careful choice of the offset used in the sampling procedure, the symmetries in k-space can be better exploited, producing higher quality reconstructions than given by standard equally-spaced samples or randomized samples motivated by compressed sensing.
Continuous K-space Recovery Network with Image Guidance for Fast MRI Reconstruction
The Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention（上海医学影像计算与计算机辅助介入重点实验室）；Academy of Engineering and Technology, Fudan University（复旦大学工程与技术学院）
Inspired by the first-order method of Malitsky and Pock, we propose a new variational framework for compressed MR image reconstruction which introduces the application of a rotation-invariant discretization of total variation functional into MR imaging while exploiting BM3D frame as a sparsifying transform. In the first step, we provide theoretical and numerical analysis establishing the exceptional rotation-invariance property of this total variation functional and observe its superiority over other well-known variational regularization terms in both upright and rotated imaging setups. Thereu…
Exploiting Completeness Perception with Diffusion Transformer for Unified 3D MRI Synthesis
School of Engineering, University of Birmingham, UK（伯明翰大学工程学院）；William Harvey Research Institute, Queen Mary University London, UK（女王玛丽大学伦敦威廉·哈里维研究所）；Barts Heart Centre, St Bartholomew’s Hospital, Barts Health NHS Trust, UK（巴特勒心脏中心，圣巴塞洛缪医院，巴特勒健康 NHS信托）；Division of Cardiology, Johns Hopkins University School of Medicine, US（约翰霍普金斯大学医学院心脏病科）
University College London（伦敦大学学院）；Technion -- Israel Institute of Technology（以色列理工学院）；Heriot-Watt University（赫瑞-瓦特大学）
Medical imaging tasks often involve multiple contrasts, such as T1- and T2-weighted magnetic resonance imaging (MRI) data. These contrasts capture information associated with the same underlying anatomy and thus exhibit similarities. In this paper, we propose a Coupled Dictionary Learning based multi-contrast MRI reconstruction (CDLMRI) approach to leverage an available guidance contrast to restore the target contrast. Our approach consists of three stages: coupled dictionary learning, coupled sparse denoising, and $k$-space consistency enforcing. The first stage learns a group of dictionaries…
BrainNormalizer: Anatomy-Informed Pseudo-Healthy Brain Reconstruction from Tumor MRI via Edge-Guided ControlNet
University of Pittsburgh（匹兹堡大学）；Georgia Institute of Technology（佐治亚理工学院）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Cedars-Sinai Medical Center（西德萨斯医疗中心）；Mayo Clinic Arizona（梅奥诊所亚利桑那分部）
Imperial College London（帝国理工学院）；University of Birmingham（伯明翰大学）；King’s College London（伦敦国王学院）
Dynamic magnetic resonance imaging (MRI) exhibits high correlations in k-space and time. In order to accelerate the dynamic MR imaging and to exploit k-t correlations from highly undersampled data, here we propose a novel deep learning based approach for dynamic MR image reconstruction, termed k-t NEXT (k-t NEtwork with X-f Transform). In particular, inspired by traditional methods such as k-t BLAST and k-t FOCUSS, we propose to reconstruct the true signals from aliased signals in x-f domain to exploit the spatio-temporal redundancies. Building on that, the proposed method then learns to recov…
Image downscaling and upscaling are two basic rescaling operations. Once the image is downscaled, it is difficult to be reconstructed via upscaling due to the loss of information. To make these two processes more compatible and improve the reconstruction performance, some efforts model them as a joint encoding-decoding task, with the constraint that the downscaled (i.e. encoded) low-resolution (LR) image must preserve the original visual appearance. To implement this constraint, most methods guide the downscaling module by supervising it with the bicubically downscaled LR version of the origin…
OPPO Research Institute（OPPO研究院）；Beihang University（北京航空航天大学）
We propose CrossHuman, a novel method that learns cross-guidance from parametric human model and multi-frame RGB images to achieve high-quality 3D human reconstruction. To recover geometry details and texture even in invisible regions, we design a reconstruction pipeline combined with tracking-based methods and tracking-free methods. Given a monocular RGB sequence, we track the parametric human model in the whole sequence, the points (voxels) corresponding to the target frame are warped to reference frames by the parametric body motion. Guided by the geometry priors of the parametric body and …
Cornell University（康奈尔大学）；University of Michigan（密歇根大学）
Electron tomography (ET) has become a standard technique for 3D characterization of materials at the nano-scale. Traditional reconstruction algorithms such as weighted back projection suffer from disruptive artifacts with insufficient projections. Popularized by compressed sensing, sparsity-exploiting algorithms have been applied to experimental ET data and show promise for improving reconstruction quality or reducing the total beam dose applied to a specimen. Nevertheless, theoretical bounds for these methods have been less explored in the context of ET applications. Here, we perform numerica…
Contrastive Diffusion Model with Auxiliary Guidance for Coarse-to-Fine PET Reconstruction
Sichuan University（四川大学）；University of Sydney（悉尼大学）；Chengdu University of Information Technology（成都信息工程大学）；ShanghaiTech University（上海科技大学）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能科技有限公司）
针对低剂量 PET 重建中扩散模型采样慢、条件对应保持不足的问题，提出由 CPM 与 IRM 构成的由粗到细框架，并引入辅助引导和对比扩散策略，提升速度、重建质量及临床可靠性。
AREA3D: Active Reconstruction Agent with Unified Feed-Forward 3D Perception and Vision-Language Guidance
Southern University of Science and Technology（南方科技大学）；Harvard University（哈佛大学）；California Institute of Technology（加州理工学院）；Massachusetts Institute of Technology（麻省理工学院）
Bio-SFT: Asymmetric Cortical Guidance and Retinal Adaptation for Robust HDR Reconstruction
College of Physics and Information Engineering, Fuzhou University（福州大学物理与信息工程学院）；School of Computer Science, Nankai University（南开大学计算机科学学院）；School of Electrical and Information Engineering, Tianjin University（天津大学电气与信息工程学院）
3D Face Reconstruction with the Geometric Guidance of Facial Part Segmentation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
The Chinese University of Hong Kong（香港中文大学）；Huya AI（虎牙AI）
We present a self-supervised learning approach to learning monocular 3D face reconstruction with a pose guidance network (PGN). First, we unveil the bottleneck of pose estimation in prior parametric 3D face learning methods, and propose to utilize 3D face landmarks for estimating pose parameters. With our specially designed PGN, our model can learn from both faces with fully labeled 3D landmarks and unlimited unlabeled in-the-wild face images. Our network is further augmented with a self-supervised learning scheme, which exploits face geometry information embedded in multiple frames of the sam…
Multi-robot autonomous 3D reconstruction using Gaussian splatting with Semantic guidance
College of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院）；College of Information Engineering, Zhejiang University of Technology（浙江工业大学信息工程学院）
NeuralDiffuser: Neuroscience-inspired Diffusion Guidance for fMRI Visual Reconstruction
Institute of Artificial Intelligence and Robotics（人工智能与机器人研究所）；Xi’an Jiaotong University（西安交通大学）；School of Electrical Engineering（电气工程学院）；Xi’an University of Technology（西安理工大学）
IPDiff: Diffusion-driven ORSI Salient Object Detection with Information Reconstruction and Multi-Prior Guidance
School of Communication and Information Engineering, Shanghai University（上海大学通信与信息工程学院）；Department of Medical Equipment, the First Affiliated Hospital of Zhengzhou University（郑州大学第一附属医院医学装备部）；School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）；Shenzhen Key Laboratory of Ubiquitous Data Enabling, Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院深圳泛在数据赋能重点实验室）
Indian Institute of Technology Kanpur（印度理工学院坎普尔分校）；Microsoft AI R&D Center（微软人工智能研发中心）；MediaTek Inc.（联发科技股份有限公司）
Despite the recent developments in 3D Face Reconstruction from occluded and noisy face images, the performance is still unsatisfactory. Moreover, most existing methods rely on additional dependencies, posing numerous constraints over the training procedure. Therefore, we propose a Self-Supervised RObustifying GUidancE (ROGUE) framework to obtain robustness against occlusions and noise in the face images. The proposed network contains 1) the Guidance Pipeline to obtain the 3D face coefficients for the clean faces and 2) the Robustification Pipeline to acquire the consistency between the estimat…
Harbin Institute of Technology（哈尔滨工业大学）；Tianjin University（天津大学）
Removing undesired reflection from an image captured through a glass surface is a very challenging problem with many practical application scenarios. For improving reflection removal, cascaded deep models have been usually adopted to estimate the transmission in a progressive manner. However, most existing methods are still limited in exploiting the result in prior stage for guiding transmission estimation. In this paper, we present a novel two-stage network with reflection-aware guidance (RAGNet) for single image reflection removal (SIRR). To be specific, the reflection layer is firstly estim…
OMGTex: One-stage Multi-style Facial Texture Reconstruction without Geometry Guidance
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）科学与工程学院）；Guangdong Provincial Key Laboratory of Future Networks of Intelligence（广东省未来网络智能化重点实验室）；FNii-Shenzhen（FNii-深圳）
China University of Petroleum（中国石油大学）；Qingdao Municipal Hospital（青岛市市立医院）；University of Chinese Academy of Sciences（中国科学院大学）
Liver cancer is one of the most common malignant diseases in the world. Segmentation and labeling of liver tumors and blood vessels in CT images can provide convenience for doctors in liver tumor diagnosis and surgical intervention. In the past decades, many state-of-the-art medical image segmentation algorithms appeared during this period. With the development of embedded devices, embedded deployment for medical segmentation and automatic reconstruction brings prospects for future automated surgical tasks. Yet, most of the existing segmentation methods mostly care about the spatial feature co…
RDG-GS: Relative Depth Guidance with Gaussian Splatting for Real-time Sparse-View 3D Rendering
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；College of Biomedical Engineering and Instrument Science, Zhejiang University（浙江大学生物医学工程与仪器科学学院）；ZJU-UIUC Institute, Zhejiang University（浙江大学-伊利诺伊大学厄巴纳-香槟分校联合学院）
Multi-view Normal and Distance Guidance Gaussian Splatting for Surface Reconstruction
The Key Laboratory of Information and Communication Systems, Ministry of Information Industry, Beijing Information Science and Technology University, Beijing 100192, China（信息与通信系统重点实验室，信息产业部，北京信息科学与技术大学，北京100192，中国）；The Key Laboratory of Target Cognition and Application Technology, Aerospace Information Research Institute, CAS, Beijing 100094, China（目标认知与应用技术重点实验室，航空信息研究所，中国科学院，北京100094，中国）
The segmentation of atrial scan images is of great significance for the three-dimensional reconstruction of the atrium and the surgical positioning. Most of the existing segmentation networks adopt a 2D structure and only take original images as input, ignoring the context information of 3D images and the role of prior information. In this paper, we propose an atrium segmentation network LGSANet with location guidance and siamese adjustment, which takes adjacent three slices of images as input and adopts an end-to-end approach to achieve coarse-to-fine atrial segmentation. The location guidanc…
