Spatial transformations are enablers in a variety of medical image analysis applications that entail aligning images to a common coordinate systems. Population analysis of such transformations is expected to capture the underlying image and shape variations, and hence these transformations are required to produce anatomically feasible correspondences. This is usually enforced through some smoothness-based generic regularization on deformation field. Alternatively, population-based regularization has been shown to produce anatomically accurate correspondences in cases where anatomically unaware (i.e., data independent) fail. Recently, deep networks have been for unsupervised image registration, these methods are computationally faster and maintains the accuracy of state of the art methods. However, these networks use smoothness penalty on deformation fields and ignores population-level statistics of the transformations. We propose a novel neural network architecture that simultaneously learns and uses the population-level statistics of the spatial transformations to regularize the neural networks for unsupervised image registration. This regularization is in the form of a bottleneck autoencoder, which encodes the population level information of the deformation fields in a low-dimensional manifold. The proposed architecture produces deformation fields that describe the population-level features and associated correspondences in an anatomically relevant manner and are statistically compact relative to the state-of-the-art approaches while maintaining computational efficiency. We demonstrate the efficacy of the proposed architecture on synthetic data sets, as well as 2D and 3D medical data.
MORPH-LER: Log-Euclidean Regularization for Population-Aware Image Registration
Karlert School of Computing, University of Utah（犹他大学卡尔尔特计算学院）；Scientific Computing and Imaging Institute, University of Utah（犹他大学科学计算与成像研究所）；Biomedical Engineering Department, University of Utah（犹他大学生物医学工程系）
LEDA: Log-Euclidean Diffeomorphism Autoencoder for Efficient Statistical Analysis of Diffeomorphisms
Scientific Computing and Imaging Institute, University of Utah（科学计算与成像研究所，犹他大学）；Kahlert School of Computing, University of Utah（Kahlert计算学院，犹他大学）；Biomedical Engineering Department, University of Utah（生物医学工程系，犹他大学）
Mask 2D-3D: Adaptive Dual-Masked Autoencoder Network for Image-to-Point Cloud Registration
Hefei University of Technology（合肥工业大学）；University of Science and Technology of China（中国科学技术大学）；Institute of Advanced Technology, University of Science and Technology of China（中国科学技术大学先进技术研究院）
Imperial College London（帝国理工学院）；University of Edinburgh（爱丁堡大学）
Image registration is an ill-posed inverse problem which often requires regularisation on the solution space. In contrast to most of the current approaches which impose explicit regularisation terms such as smoothness, in this paper we propose a novel method that can implicitly learn biomechanics-informed regularisation. Such an approach can incorporate application-specific prior knowledge into deep learning based registration. Particularly, the proposed biomechanics-informed regularisation leverages a variational autoencoder (VAE) to learn a manifold for biomechanically plausible deformations…
National Cheng Kung University（成功大学）；Umeå University（于默奥大学）
Facial feature tracking is a key component of imaging ballistocardiography (BCG) where accurate quantification of the displacement of facial keypoints is needed for good heart rate estimation. Skin feature tracking enables video-based quantification of motor degradation in Parkinson's disease. Traditional computer vision algorithms include Scale Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), and Lucas-Kanade method (LK). These have long represented the state-of-the-art in efficiency and accuracy but fail when common deformations, like affine local transformations or ill…
Due to their inference, data representation and reconstruction properties, Variational Autoencoders (VAE) have been successfully used in continual learning classification tasks. However, their ability to generate images with specifications corresponding to the classes and databases learned during Continual Learning (CL) is not well understood and catastrophic forgetting remains a significant challenge. In this paper, we firstly analyze the forgetting behaviour of VAEs by developing a new theoretical framework that formulates CL as a dynamic optimal transport problem. This framework proves appr…
Hefei University（合肥学院）；School of Artificial Intelligence and Big Data, Hefei University（合肥学院人工智能与大数据学院）；Institute of Applied Optimization, School of Artificial Intelligence and Big Data, Hefei University（合肥学院人工智能与大数据学院应用优化研究所）；School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；University of Science and Technology of China（中国科学技术大学）
Recently, with the availability of cost-effective depth cameras coupled with real-time skeleton estimation, the interest in skeleton-based human action recognition is renewed. Most of the existing skeletal representation approaches use either the joint location or the dynamics model. Differing from the previous studies, we propose a new method called Denoising Autoencoder with Temporal and Categorical Constraints (DAE_CTC)} to study the skeletal representation in a view of skeleton reconstruction. Based on the concept of learning under privileged information, we integrate action categories and…
Masked Registration and Autoencoding of CT Images for Predictive Tibia Reconstruction
Technical University Berlin（技术大学柏林）；Charité – Universitätsmedizin Berlin（柏林查理医院）；Freie Universität Berlin（自由大学柏林）；Humboldt-Universität zu Berlin（洪堡大学柏林）；Institute of Medical Informatics（医学信息学研究所）；Center for Musculoskeletal Surgery（骨科手术中心）；Julius Wolff Institute（朱利叶斯·沃尔夫研究所）
Ulsan National Institute of Science and Technology（蔚山科学技术院）；The Rockefeller University（洛克菲勒大学）；Harvard Medical School（哈佛医学院）
The alignment of serial-section electron microscopy (ssEM) images is critical for efforts in neuroscience that seek to reconstruct neuronal circuits. However, each ssEM plane contains densely packed structures that vary from one section to the next, which makes matching features across images a challenge. Advances in deep learning has resulted in unprecedented performance in similar computer vision problems, but to our knowledge, they have not been successfully applied to ssEM image co-registration. In this paper, we introduce a novel deep network model that combines a spatial transformer for …
We propose a deformable registration algorithm based on unsupervised learning of a low-dimensional probabilistic parameterization of deformations. We model registration in a probabilistic and generative fashion, by applying a conditional variational autoencoder (CVAE) network. This model enables to also generate normal or pathological deformations of any new image based on the probabilistic latent space. Most recent learning-based registration algorithms use supervised labels or deformation models, that miss important properties such as diffeomorphism and sufficiently regular deformation field…
University of Edinburgh（爱丁堡大学）；Fudan University（复旦大学）；Imperial College London（帝国理工学院）；Technical University of Munich（慕尼黑工业大学）
Myocardial motion and deformation are rich descriptors that characterize cardiac function. Image registration, as the most commonly used technique for myocardial motion tracking, is an ill-posed inverse problem which often requires prior assumptions on the solution space. In contrast to most existing approaches which impose explicit generic regularization such as smoothness, in this work we propose a novel method that can implicitly learn an application-specific biomechanics-informed prior and embed it into a neural network-parameterized transformation model. Particularly, the proposed method …
Inception Institute of Artificial Intelligence（人工智能 inception 研究所）
Medical image registration is an important task in automated analysis of multi-modal images and temporal data involving multiple patient visits. Conventional approaches, although useful for different image types, are time consuming. Of late, deep learning (DL) based image registration methods have been proposed that outperform traditional methods in terms of accuracy and time. However,DL based methods are heavily dependent on training data and do not generalize well when presented with images of different scanners or anatomies. We present a DL based approach that can perform medical image regi…
Medical Imaging Research Center, UZ Leuven（UZ Leuven医学成像研究中心）；Department of Electrical Engineering, ESAT/PSI, KU Leuven（KU Leuven电气工程系（ESAT/PSI））；Center for Craniofacial and Dental Genetics, Department of Oral and Craniofacial Sciences, University of Pittsburgh（匹兹堡大学口腔与颅面科学系颅面和牙科遗传学中心）；Department of Computing, Imperial College London（帝国理工学院计算机系）；IDSIA, USI Lugano（卢加诺大学IDSIA研究所）；Twitter（推特公司）；Department of Human Genetics, KU Leuven（KU Leuven人类遗传学系）
Registration is an essential tool in image analysis. Deep learning based alternatives have recently become popular, achieving competitive performance at a faster speed. However, many contemporary techniques are limited to volumetric representations, despite increased popularity of 3D surface and shape data in medical image analysis. We propose a one-step registration model for 3D surfaces that internalises a lower dimensional probabilistic deformation model (PDM) using conditional variational autoencoders (CVAE). The deformations are constrained to be diffeomorphic using an exponentiation laye…
We propose to learn a low-dimensional probabilistic deformation model from data which can be used for registration and the analysis of deformations. The latent variable model maps similar deformations close to each other in an encoding space. It enables to compare deformations, generate normal or pathological deformations for any new image or to transport deformations from one image pair to any other image. Our unsupervised method is based on variational inference. In particular, we use a conditional variational autoencoder (CVAE) network and constrain transformations to be symmetric and diffe…
the University of Sydney（悉尼大学）；Royal Prince Alfred Hospital（皇家阿尔弗雷德王子医院）；Shanghai Jiao Tong University（上海交通大学）
Deformable image registration is fundamental for many medical image analyses. A key obstacle for accurate image registration lies in image appearance variations such as the variations in texture, intensities, and noise. These variations are readily apparent in medical images, especially in brain images where registration is frequently used. Recently, deep learning-based registration methods (DLRs), using deep neural networks, have shown computational efficiency that is several orders of magnitude faster than traditional optimization-based registration methods (ORs). DLRs rely on a globally opt…
Enhancing Deformable Local Features by Jointly Learning to Detect and Describe Keypoints
Universidade Federal de Minas Gerais（米纳斯吉拉斯联邦大学）；Google Research（谷歌研究院）；Université de Bourgogne（勃艮第大学）；Université de Lorraine（洛林大学）；Inria（法国国家信息与自动化研究所）；Microsoft（微软）
