This article introduces a new physics-based method for rigid point set alignment called Fast Gravitational Approach (FGA). In FGA, the source and target point sets are interpreted as rigid particle swarms with masses interacting in a globally multiply-linked manner while moving in a simulated gravitational force field. The optimal alignment is obtained by explicit modeling of forces acting on the particles as well as their velocities and displacements with second-order ordinary differential equations of motion. Additional alignment cues (point-based or geometric features, and other boundary conditions) can be integrated into FGA through particle masses. We propose a smooth-particle mass function for point mass initialization, which improves robustness to noise and structural discontinuities. To avoid prohibitive quadratic complexity of all-to-all point interactions, we adapt a Barnes-Hut tree for accelerated force computation and achieve quasilinear computational complexity. We show that the new method class has characteristics not found in previous alignment methods such as efficient handling of partial overlaps, inhomogeneous point sampling densities, and coping with large point clouds with reduced runtime compared to the state of the art. Experiments show that our method performs on par with or outperforms all compared competing non-deep-learning-based and general-purpose techniques (which do not assume the availability of training data and a scene prior) in resolving transformations for LiDAR data and gains state-of-the-art accuracy and speed when coping with different types of data disturbances.
We can use a method called registration to integrate some point clouds that represent the shape of the real world. In this paper, we propose highly accurate and stable registration method. Our method detects keypoints from point clouds and generates triplets using multiple descriptors. Furthermore, our method evaluates the consistency of rigid transformation parameters of each triplet with histograms and obtains the rigid transformation between the point clouds. In the experiment of this paper, our method had minimul errors and no major failures. As a result, we obtained sufficiently accurate …
National University of Defense Technology（国防科技大学）；Technical University of Munich（慕尼黑工业大学）
We study the problem of outlier correspondence pruning for non-rigid point cloud registration. In rigid registration, spatial consistency has been a commonly used criterion to discriminate outliers from inliers. It measures the compatibility of two correspondences by the discrepancy between the respective distances in two point clouds. However, spatial consistency no longer holds in non-rigid cases and outlier rejection for non-rigid registration has not been well studied. In this work, we propose Graph-based Spatial Consistency Network (GraphSCNet) to filter outliers for non-rigid registratio…
Non-rigid point cloud registration is a key component in many computer vision and computer graphics applications. The high complexity of the unknown non-rigid motion make this task a challenging problem. In this paper, we break down this problem via hierarchical motion decomposition. Our method called Neural Deformation Pyramid (NDP) represents non-rigid motion using a pyramid architecture. Each pyramid level, denoted by a Multi-Layer Perception (MLP), takes as input a sinusoidally encoded 3D point and outputs its motion increments from the previous level. The sinusoidal function starts with a…
In this paper, we unify popular non-rigid registration methods for point sets and surfaces under our general framework, GiNGR. GiNGR builds upon Gaussian Process Morphable Models (GPMM) and hence separates modeling the deformation prior from model adaptation for registration. In addition, it provides explainable hyperparameters, multi-resolution registration, trivial inclusion of expert annotation, and the ability to use and combine analytical and statistical deformation priors. But more importantly, the reformulation allows for a direct comparison of registration methods. Instead of using a g…
The Discriminative Optimization (DO) algorithm has been proved much successful in 3D point cloud registration. In the original DO, the feature (descriptor) of two point cloud was defined as a histogram, and the element of histogram indicates the weights of scene points in "front" or "back" side of a model point. In this paper, we extended the histogram which indicate the sides from "front-back" to "front-back", "up-down", and "clockwise-anticlockwise". In addition, we reweighted the extended histogram according to the model points' distribution. We evaluated the proposed Improved DO on the Sta…
Point Cloud Registration is the problem of aligning the corresponding points of two 3D point clouds referring to the same object. The challenges include dealing with noise and partial match of real-world 3D scans. For non-rigid objects, there is an additional challenge of accounting for deformations in the object shape that happen to the object in between the two 3D scans. In this project, we study the problem of non-rigid point cloud registration for use cases in the Augmented/Mixed Reality domain. We focus our attention on a special class of non-rigid deformations that happen in rigid object…
SemICP: Semantic Non-Rigid Point Cloud Registration with Elastic Energy Regularization
Department of Electrical and Computer Engineering, University of British Columbia（电气与计算机工程系，不列颠哥伦比亚大学）；Faculty of Medicine, University of British Columbia（医学院，不列颠哥伦比亚大学）
Laboratório Nacional de Computação Científica（国家计算科学实验室）；Universidade Federal de Juiz de Fora（茹伊斯迪福拉联邦大学）
In the last decades, some literature appeared using the Lie groups theory to solve problems in computer vision. On the other hand, Lie algebraic representations of the transformations therein were introduced to overcome the difficulties behind group structure by mapping the transformation groups to linear spaces. In this paper we focus on application of Lie groups and Lie algebras to find the rigid transformation that best register two surfaces represented by point clouds. The so called pairwise rigid registration can be formulated by comparing intrinsic second-order orientation tensors that e…
Inria（法国国家信息与自动化研究所）；Mines ParisTech（巴黎矿业学院）；PSL Research University（巴黎文理研究大学）；LIGM（巴黎马恩河谷信息、数学与系统实验室）；Ecole des Ponts（巴黎路桥学院）；Univ Gustave Eiffel（古斯塔夫·埃菲尔大学）；CNRS（法国国家科学研究中心）
Rigid registration of point clouds with partial overlaps is a longstanding problem usually solved in two steps: (a) finding correspondences between the point clouds; (b) filtering these correspondences to keep only the most reliable ones to estimate the transformation. Recently, several deep nets have been proposed to solve these steps jointly. We built upon these works and propose PCAM: a neural network whose key element is a pointwise product of cross-attention matrices that permits to mix both low-level geometric and high-level contextual information to find point correspondences. These cro…
TU Kaiserslautern（凯泽斯劳滕工业大学）；German Research Center for Artificial Intelligence (DFKI GmbH)（德国人工智能研究中心（DFKI GmbH））
We propose RPSRNet - a novel end-to-end trainable deep neural network for rigid point set registration. For this task, we use a novel $2^D$-tree representation for the input point sets and a hierarchical deep feature embedding in the neural network. An iterative transformation refinement module in our network boosts the feature matching accuracy in the intermediate stages. We achieve an inference speed of 12-15ms to register a pair of input point clouds as large as 250K. Extensive evaluation on (i) KITTI LiDAR odometry and (ii) ModelNet-40 datasets shows that our method outperforms prior state…
School of Engineering, The University of Tokyo（东京大学工学部）
Traditional algorithms of point set registration minimizing point-to-plane distances often achieve a better estimation of rigid transformation than those minimizing point-to-point distances. Nevertheless, recent deep-learning-based methods minimize the point-to-point distances. In contrast to these methods, this paper proposes the first deep-learning-based approach to point-to-plane registration. A challenging part of this problem is that a typical solution for point-to-plane registration requires an iterative process of accumulating small transformations obtained by minimizing a linearized en…
Depth Augmented and FE Free 3D/2D Liver Registration for Laparoscopic Liver AR
UCL Hawkes Institute, University College London, London WC1E 6BT, UK（伦敦大学学院UCL哈维斯研究所）；Division of Surgery and Interventional Science, University College London, London WC1E 6BT, UK（伦敦大学学院UCL外科与介入科学系）；Unit for Lifelong Health and Ageing at UCL, University College London, London WC1E 7HB, UK（伦敦大学学院UCL终身健康与老龄化单位）；Medtronic plc., London, UK（伦敦梅脱利克公司）
We present SyNoRiM, a novel way to jointly register multiple non-rigid shapes by synchronizing the maps relating learned functions defined on the point clouds. Even though the ability to process non-rigid shapes is critical in various applications ranging from computer animation to 3D digitization, the literature still lacks a robust and flexible framework to match and align a collection of real, noisy scans observed under occlusions. Given a set of such point clouds, our method first computes the pairwise correspondences parameterized via functional maps. We simultaneously learn potentially n…
Matching 3D rigid point clouds in complex environments robustly and accurately is still a core technique used in many applications. This paper proposes a new architecture combining error estimation from sample covariances and dual global probability alignment based on the convolution of adaptive Gaussian Mixture Models (GMM) from point clouds. Firstly, a novel adaptive GMM is defined using probability distributions from the corresponding points. Then rigid point cloud alignment is performed by maximizing the global probability from the convolution of dual adaptive GMMs in the whole 2D or 3D sp…
SynBench: A Synthetic Benchmark for Non-rigid 3D Point Cloud Registration
Mannheim Institute for Intelligent Systems in Medicine (MIISM), Medical Faculty Mannheim, Heidelberg University, Mannheim 68167, Germany（海德堡大学曼海姆医学院智能医学研究所（MIISM））；Department of Radiation Oncology, Brigham and Women’s Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, USA（放射肿瘤科，布里格斯妇女医院，达纳-法伯癌症研究所，哈佛医学院，马萨诸塞州波士顿，美国）；Interdisciplinary Center for Scientific Computing (IWR), Heidelberg University, Heidelberg, Germany（科学计算跨学科中心（IWR），海德堡大学，海德堡，德国）；Central Institute for Computer Engineering (ZITI), Heidelberg University, Heidelberg, Germany（计算机工程中央研究所（ZITI），海德堡大学，海德堡，德国）；CZS Heidelberg Center for Model-Based AI, Heidelberg University, Mannheim, Germany（海德堡模型驱动人工智能中心（CZS），海德堡大学，曼海姆，德国）；Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital Mannheim, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany（耳鼻喉科及头颈外科，曼海姆大学医院，曼海姆医学院，海德堡大学，曼海姆，德国）；AI Health Innovation Cluster, Heidelberg-Mannheim Health and Life Science Alliance, Heidelberg University, Heidelberg, Germany（人工智能健康创新集群，海德堡-曼海姆健康与生命科学联盟，海德堡大学，海德堡，德国）
Point cloud registration is a key problem for computer vision applied to robotics, medical imaging, and other applications. This problem involves finding a rigid transformation from one point cloud into another so that they align. Iterative Closest Point (ICP) and its variants provide simple and easily-implemented iterative methods for this task, but these algorithms can converge to spurious local optima. To address local optima and other difficulties in the ICP pipeline, we propose a learning-based method, titled Deep Closest Point (DCP), inspired by recent techniques in computer vision and n…
Occlusion-aware Non-Rigid Point Cloud Registration via Unsupervised Neural Deformation Correntropy
Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Deep learning based Non-Rigid Volume-to-Surface Registration for Brain Shift compensation Using Point Cloud
Austrian Center for Medical Innovation and Technology（奥地利医学创新与技术中心）；Medical University of Vienna（维也纳医学大学）；University of Cambridge（剑桥大学）；Danube Private University (DPU)（多瑙私人大学（DPU））
University of Maryland, College Park（马里兰大学帕克分校）；University of Maryland Institute for Advanced Computer Studies (UMIACS), University of Maryland, College Park（马里兰大学帕克分校马里兰高级计算机研究所）
In this paper, we introduce a non-rigid registration pipeline for pairs of unorganized point clouds that may be topologically different. Standard warp field estimation algorithms, even under robust, discontinuity-preserving regularization, tend to produce erratic motion estimates on boundaries associated with `close-to-open' topology changes. We overcome this limitation by exploiting backward motion: in the opposite motion direction, a `close-to-open' event becomes `open-to-close', which is by default handled correctly. At the core of our approach lies a general, topology-agnostic warp field e…
DefTransNet: A Transformer-based Method for Non-Rigid Point Cloud Registration in the Simulation of Soft Tissue Deformation
Mannheim Institute for Intelligent Systems in Medicine (MIISM), Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany（曼海姆智能医学研究所（MIISM）、曼海姆医学院、海德堡大学）；Department of Radiation Oncology, Brigham and Women's Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, USA（放射肿瘤科、布里格姆和女性医院、达纳-法伯癌症研究所、哈佛医学院）；International Institute of Information Technology, Bangalore, India（国际信息技术研究所，班加罗尔，印度）；Department of Otorhinolaryngology, Head and Neck Surgery, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany（耳鼻喉科与头颈外科、曼海姆医学院、海德堡大学）；Interdisciplinary Center for Scientific Computing (IWR), Heidelberg University, Heidelberg, Germany（跨学科科学计算中心（IWR）、海德堡大学）；Central Institute for Computer Engineering (ZITI), Heidelberg University, Heidelberg, Germany（计算机工程研究中心（ZITI）、海德堡大学）；CZS Heidelberg Center for Model-Based AI, Heidelberg University, Mannheim, Germany（海德堡模型驱动人工智能中心（CZS）、海德堡大学）
Northwestern Polytechnical University（西北工业大学）；Baidu Research（百度研究院）；National Engineering Laboratory of Deep Learning Technology and Application（深度学习技术及应用国家工程实验室）
3D point cloud registration in remote sensing field has been greatly advanced by deep learning based methods, where the rigid transformation is either directly regressed from the two point clouds (correspondences-free approaches) or computed from the learned correspondences (correspondences-based approaches). Existing correspondences-free methods generally learn the holistic representation of the entire point cloud, which is fragile for partial and noisy point clouds. In this paper, we propose a correspondences-free unsupervised point cloud registration (UPCR) method from the representation se…
Image registration algorithms can be generally categorized into two groups: non-rigid and rigid. Recently, many deep learning-based algorithms employ a neural net to characterize non-rigid image registration function. However, do they always perform better? In this study, we compare the state-of-art deep learning-based non-rigid registration approach with rigid registration approach. The data is generated from Kaggle Dog vs Cat Competition \url{https://www.kaggle.com/c/dogs-vs-cats/} and we test the algorithms' performance on rigid transformation including translation, rotation, scaling, shear…
Iterative Closest Point (ICP) solves the rigid point cloud registration problem iteratively in two steps: (1) make hard assignments of spatially closest point correspondences, and then (2) find the least-squares rigid transformation. The hard assignments of closest point correspondences based on spatial distances are sensitive to the initial rigid transformation and noisy/outlier points, which often cause ICP to converge to wrong local minima. In this paper, we propose the RPM-Net -- a less sensitive to initialization and more robust deep learning-based approach for rigid point cloud registrat…
The main goal of point cloud registration in Multi-View Partial (MVP) Challenge 2021 is to estimate a rigid transformation to align a point cloud pair. The pairs in this competition have the characteristics of low overlap, non-uniform density, unrestricted rotations and ambiguity, which pose a huge challenge to the registration task. In this report, we introduce our solution to the registration task, which fuses two deep learning models: ROPNet and PREDATOR, with customized ensemble strategies. Finally, we achieved the second place in the registration track with 2.96546, 0.02632 and 0.07808 un…
National University of Singapore（新加坡国立大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Probabilistic point cloud registration methods are becoming more popular because of their robustness. However, unlike point-to-plane variants of iterative closest point (ICP) which incorporate local surface geometric information such as surface normals, most probabilistic methods (e.g., coherent point drift (CPD)) ignore such information and build Gaussian mixture models (GMMs) with isotropic Gaussian covariances. This results in sphere-like GMM components which only penalize the point-to-point distance between the two point clouds. In this paper, we propose a novel method called CPD with Loca…
Purpose: In laparoscopic liver surgery (LLS), pre-operative information can be overlaid onto the intra-operative scene by registering a 3D pre-operative model to the intra-operative partial surface reconstructed from the laparoscopic video. To assist with this task, we explore the use of learning-based feature descriptors, which, to our best knowledge, have not been explored for use in laparoscopic liver registration. Furthermore, a dataset to train and evaluate the use of learning-based descriptors does not exist. Methods: We present the LiverMatch dataset consisting of 16 preoperative models…
University of Michigan（密歇根大学）；Toyota Research Institute（丰田研究院）；Cornell University（康奈尔大学）
This paper reports on a novel nonparametric rigid point cloud registration framework that jointly integrates geometric and semantic measurements such as color or semantic labels into the alignment process and does not require explicit data association. The point clouds are represented as nonparametric functions in a reproducible kernel Hilbert space. The alignment problem is formulated as maximizing the inner product between two functions, essentially a sum of weighted kernels, each of which exploits the local geometric and semantic features. As a result of the continuous models, analytical gr…
We provide a dynamical perspective on the classical problem of 3D point cloud registration with correspondences. A point cloud is considered as a rigid body consisting of particles. The problem of registering two point clouds is formulated as a dynamical system, where the dynamic model point cloud translates and rotates in a viscous environment towards the static scene point cloud, under forces and torques induced by virtual springs placed between each pair of corresponding points. We first show that the potential energy of the system recovers the objective function of the maximum likelihood e…
GMOR: A Lightweight Robust Point Cloud Registration Framework via Geometric Maximum Overlapping
Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology（北京混合现实与先进显示工程研究中心，光学与 photonics 学院，北京理工大学）；Zhengzhou Research Institute, Beijing Institute of Technology（郑州研究院，北京理工大学）；School of Computer Science and Technology, Beijing Institute of Technology（计算机科学与技术学院，北京理工大学）；School of Medical Technology, Beijing Institute of Technology（医学技术学院，北京理工大学）
Geo-Registration of Terrestrial LiDAR Point Clouds with Satellite Images without GNSS
Department of Computer Science and Software Engineering, The University of Western Australia（计算机科学与软件工程系，西澳大学）；School of Engineering and Technology, University of New South Wales（工程与技术学院，新南威尔士大学）
Images taken at different times or positions undergo transformations such as rotation, scaling, skewing, and more. The process of aligning different images which have undergone transformations can be done via registration. Registration is desirable when analyzing time-series data for tracking, averaging, or differential diagnoses of diseases. Efficient registration methods exist for rigid (including linear or affine) transformations; however, for non-rigid (also known as non-affine) transformations, current methods are computationally expensive and time-consuming. In this report, I will explor…
The non-rigid registration between CT data and ultrasonic images of liver can facilitate the diagnosis and treatment, which has been widely studied in recent years. To improve the registration accuracy of the Demons model on the non-rigid registration between 3D CT liver data and 2D ultrasonic images, a novel boundary extraction and enhancement method based on radial directional local intuitionistic fuzzy entropy in the polar coordinates has been put forward, and a new registration workflow has been provided. Experiments show that our method can acquire high-accuracy registration results. Expe…
Learning accurate rigid registration for longitudinal brain MRI from synthetic data
1 Division of Biomedical Imaging, KTH Royal Institute of Technology, Huddinge, Sweden 2 Athinoula A.\ Martinos Center for Biomedical Imaging, Charlestown, USA 3 Department of Radiology, Massachusetts General Hospital, Boston, USA 4 Department of Radiology, Harvard Medical School, Boston, USA 5 Computer Science \& Artificial Intelligence Laboratory, MIT, Cambridge, USA
Robust Rigid and Non-Rigid Medical Image Registration Using Learnable Edge Kernels
Department of Mathematics, University of Innsbruck（因斯布鲁克大学数学系）；Department of Radiology, Medical University of Innsbruck（因斯布鲁克医学大学放射学系）；Neuroimaging Research Core Facility, Medical University of Innsbruck（因斯布鲁克医学大学神经影像研究核心设施）
This paper addresses the issue of matching rigid 3D object points with 2D image points through point registration based on maximum likelihood principle in computer simulated images. Perspective projection is necessary when transforming 3D coordinate into 2D. The problem then recasts into a missing data framework where unknown correspondences are handled via mixture models. Adopting the Expectation Conditional Maximization for Point Registration (ECMPR), two different rotation and translation optimization algorithms are compared in this paper. We analyze in detail the associated consequences in…
An Optimization-based Baseline for Rigid 2D/3D Registration Applied to Spine Surgical Navigation Using CMA-ES
School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Jiangsu Provincial Joint International Research Laboratory of Medical Information Processing, Southeast University（东南大学江苏省医学信息处理协同创新国际联合实验室）；Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education（教育部新一代人工智能技术及其交叉应用重点实验室（东南大学））
UDCR: Unsupervised Aortic DSA/CTA Rigid Registration Using Deep Reinforcement Learning and Overlap Degree Calculation
School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）；Shanghai Changhai Hospital, Naval Medical University（海军军医大学上海长海医院）；Dalian University of Technology（大连理工大学）；School of Computer Science and Information Security, Guilin University of Electronic Technology（桂林电子科技大学计算机科学与信息安全学院）；The Fourth Hospital of Hebei Medical University and Hebei Tumor Hospital（河北医科大学第四医院暨河北省肿瘤医院）；Erasmus MC, University Medical Center Rotterdam（鹿特丹伊拉斯姆斯大学医学中心）
We present a new fully-automatic non-rigid 3D shape registration (morphing) framework comprising (1) a new 3D landmarking and pose normalisation method; (2) an adaptive shape template method to accelerate the convergence of registration algorithms and achieve a better final shape correspondence and (3) a new iterative registration method that combines Iterative Closest Points with Coherent Point Drift (CPD) to achieve a more stable and accurate correspondence establishment than standard CPD. We call this new morphing approach Iterative Coherent Point Drift (ICPD). Our proposed framework is eva…
