The aim of this paper is to derive and analyze a variational model for the joint estimation of motion and reconstruction of image sequences, which is based on a time-continuous Eulerian motion model. The model can be set up in terms of the continuity equation or the brightness constancy equation. The analysis in this paper focuses on the latter for robust motion estimation on sequences of two-dimensional images. We rigorously prove the existence of a minimizer in a suitable function space setting. Moreover, we discuss the numerical solution of the model based on primal-dual algorithms and investigate several examples. Finally, the benefits of our model compared to existing techniques, such as sequential image reconstruction and motion estimation, are shown.
A New Variational Model for Joint Image Reconstruction and Motion Estimation in Spatiotemporal Imaging
LSEC, ICMSEC, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（LSEC，ICMSEC，数学系统科学研究所，中国科学院）；Department of Mathematics, KTH–Royal Institute of Technology（数学系，皇家理工学院）
This article describes the implementation of the joint motion estimation and image reconstruction framework presented by Burger, Dirks and Schönlieb and extends this framework to large-scale motion between consecutive image frames. The variational framework uses displacements between consecutive frames based on the optical flow approach to improve the image reconstruction quality on the one hand and the motion estimation quality on the other. The energy functional consists of a data-fidelity term with a general operator that connects the input sequence to the solution, it has a total variation…
A Variational Reconstruction Method for Undersampled Dynamic X-ray Tomography based on Physical Motion Models
Department of Computer Science, University College London（伦敦大学学院计算机科学系）；Department of Mathematics and Statistics, University of Helsinki（赫尔辛基大学数学与统计学系）
Motion degradation is a central problem in Magnetic Resonance Imaging (MRI). This work addresses the problem of how to obtain higher quality, super-resolved motion-free, reconstructions from highly undersampled MRI data. In this work, we present for the first time a variational multi-task framework that allows joining three relevant tasks in MRI: reconstruction, registration and super-resolution. Our framework takes a set of multiple undersampled MR acquisitions corrupted by motion into a novel multi-task optimisation model, which is composed of an $L^2$ fidelity term that allows sharing repre…
A Nonlinear Variational Approach to Motion-Corrected Reconstruction of Density Images
University of Münster（明斯特大学）；Institute for Computational and Applied Mathematics, University of Münster（明斯特大学计算与应用数学研究所）；Cells in Motion Cluster of Excellence, University of Münster（明斯特大学运动细胞卓越集群）；University of Lübeck（吕贝克大学）；Institute of Mathematics and Image Computing, University of Lübeck（吕贝克大学数学与图像计算研究所）；Fraunhofer MEVIS Project Group Image Registration（弗劳恩霍夫 MEVIS 图像配准项目组）
Huawei Noah’s Ark Lab, Huawei Technologies（华为诺亚方舟实验室，华为）；University College London（伦敦大学学院）
Learned image reconstruction techniques using deep neural networks have recently gained popularity, and have delivered promising empirical results. However, most approaches focus on one single recovery for each observation, and thus neglect the uncertainty information. In this work, we develop a novel computational framework that approximates the posterior distribution of the unknown image at each query observation. The proposed framework is very flexible: It handles implicit noise models and priors, it incorporates the data formation process (i.e., the forward operator), and the learned recon…
The paper surveys variational approaches for image reconstruction in dynamic inverse problems. Emphasis is on methods that rely on parametrised temporal models. These are here encoded as diffeomorphic deformations with time dependent parameters, or as motion constrained reconstruction where the motion model is given by a partial differential equation. The survey also includes recent development in integrating deep learning for solving these computationally demanding variational methods. Examples are given for 2D dynamic tomography, but methods apply to general inverse problems.
PET-MRI Joint Reconstruction by Joint Sparsity Based Tight Frame Regularization
Institute of Natural Sciences, Shanghai Jiao Tong University（上海交通大学自然科学研究院）；Yau Mathematical Sciences Center, Tsinghua University（清华大学丘成桐数学科学中心）；Institute of Natural Sciences, School of Mathematical Sciences, and MOE-LSC, Shanghai Jiao Tong University（上海交通大学自然科学研究院、数学科学学院和教育部计算数学重点实验室）
Non-uniqueness and instability are characteristic features of image reconstruction processes. As a result, it is necessary to develop regularization methods that can be used to compute reliable approximate solutions. A regularization method provides of a family of stable reconstructions that converge to an exact solution of the noise-free problem as the noise level tends to zero. The standard regularization technique is defined by variational image reconstruction, which minimizes a data discrepancy augmented by a regularizer. The actual numerical implementation makes use of iterative methods, …
A primary interest in dynamic inverse problems is to identify the underlying temporal behaviour of the system from outside measurements. In this work we consider the case, where the target can be represented by a decomposition of spatial and temporal basis functions and hence can be efficiently represented by a low-rank decomposition. We then propose a joint reconstruction and low-rank decomposition method based on the Nonnegative Matrix Factorisation to obtain the unknown from highly undersampled dynamic measurement data. The proposed framework allows for flexible incorporation of separate re…
One important property of imaging modalities and related applications is the resolution of image reconstructions which relies on various factors such as instrumentation or data processing. Restrictions in resolution can have manifold origins, e.g., limited resolution of available data, noise level in the data, and/or inexact model operators. In this work we investigate a novel data processing approach suited for inexact model operators. Here, two different information sources, high-dimensional model information and high-quality measurement on a lower resolution, are comprised in a hybrid appro…
Position-Blind Ptychography: Viability of image reconstruction via data-driven variational inference
Department of Informatics, University of Hamburg（汉堡大学信息学院）；Center for Free-Electron Laser Science CFEL, Deutsches Elektronen-Synchrotron DESY（自由电子激光科学中心 CFEL，德意志电子同步辐射实验室）；Department of Mathematics, Bundesstr. 55, University of Hamburg（汉堡大学数学系）；CIT School, Technical University of Munich（慕尼黑技术大学 CIT 学院）；Munich Center for Machine Learning, München（慕尼黑机器学习中心）；Massachusetts Institute of Technology (MIT)（麻省理工学院）；The NSF AI Institute for Artificial Intelligence and Fundamental Interactions（国家科学基金会人工智能与基本相互作用研究院）；Helmholtz Imaging, Deutsches Elektronen-Synchrotron DESY（海德堡成像，德意志电子同步辐射实验室）
