Diff-PCR: Diffusion-Based Correspondence Searching in Doubly Stochastic Matrix Space for Point Cloud Registration
Jinling Institute of Technology（金陵科技学院）；PCA Lab, Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, and Jiangsu Key Lab of Image and Video Understanding for Social Security, School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院，教育部高维信息智能感知与系统重点实验室，江苏省社会安全图像与视频理解重点实验室，PCA实验室）
高效地识别点云之间的准确对应关系对于刚性和非刚性点云配准至关重要。现有方法通常依赖几何或语义特征嵌入来建立对应关系并估计变换或流场。最近，一些最先进的方法采用了RAFT-like迭代更新来优化解决方案。然而，这些方法仍有两个主要局限。首先，其迭代优化机制缺乏透明性，一旦开始优化，更新轨迹大多固定，可能导致次优解。其次，它们忽略了在求解变换或流场之前显式优化对应矩阵的重要性。大多数现有方法在特征空间中计算候选对应关系，并仅通过Sinkhorn或双softmax归一化一次投影得到匹配矩阵。这种一次性投影可能远离全局最优解，且这些方法通常不建模目标匹配矩阵的分布。在本文中，我们提出了一种新的框架，利用去噪扩散模型来预测双随机矩阵空间中最优匹配矩阵的搜索梯度。具体来说，扩散模型学习了去噪方向，反向去噪过程沿此学习方向迭代搜索改进的解决方案，近似了目标匹配矩阵的最大似然方向。为了提高效率，我们设计了一个轻量级去噪模块，并采用了Denoising Diffusion Implicit Model (DDIM)\cite{song2020denoising}的加速采样策略。在3DMatch/3DLoMatch和4DMatch/4DLoMatch上的实验结果验证了所提框架的有效性。
英文摘要：
Efficiently identifying accurate correspondences between point clouds is crucial for both rigid and non-rigid point cloud registration. Existing methods usually rely on geometric or semantic feature embeddings to establish correspondences and then estimate transformations or flow fields. Recently, several state-of-the-art methods have adopted RAFT-like iterative updates to refine solutions. However, these methods still have two major limitations. First, their iterative refinement mechanism lacks transparency, and the update trajectory is largely fixed once the refinement starts, which may lead to suboptimal solutions. Second, they overlook the importance of explicitly refining the correspondence matrix before solving for transformations or flow fields. Most existing approaches compute candidate correspondences in feature space and project the resulting matching matrix only once by using Sinkhorn or dual-softmax normalization. Such a one-shot projection can be far from the globally optimal solution, and these methods usually do not model the distribution of the target matching matrix. In this paper, we propose a novel framework that exploits a denoising diffusion model to predict a search gradient for the optimal matching matrix in doubly stochastic matrix space. Specifically, the diffusion model learns a denoising direction, and the reverse denoising process iteratively searches for improved solutions along this learned direction, which approximates the maximum-likelihood direction of the target matching matrix. To improve efficiency, we design a lightweight denoising module and adopt the accelerated sampling strategy of the Denoising Diffusion Implicit Model (DDIM)\cite{song2020denoising}. Experimental results on 3DMatch/3DLoMatch and 4DMatch/4DLoMatch demonstrate the effectiveness of the proposed framework.
Huazhong University of Science and Technology（华中科技大学）；China University of Geosciences（中国地质大学）
In this paper, we present a second order spatial compatibility (SC^2) measure based method for efficient and robust point cloud registration (PCR), called SC^2-PCR. Firstly, we propose a second order spatial compatibility (SC^2) measure to compute the similarity between correspondences. It considers the global compatibility instead of local consistency, allowing for more distinctive clustering between inliers and outliers at early stage. Based on this measure, our registration pipeline employs a global spectral technique to find some reliable seeds from the initial correspondences. Then we des…
Sight View Constraint for Robust Point Cloud Registration
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Guangdong Provincial Key Laboratory of Construction Robotics and Intelligent Construction（广东省建造机器人与智能建造重点实验室）
PointNet has recently emerged as a popular representation for unstructured point cloud data, allowing application of deep learning to tasks such as object detection, segmentation and shape completion. However, recent works in literature have shown the sensitivity of the PointNet representation to pose misalignment. This paper presents a novel framework that uses the PointNet representation to align point clouds and perform registration for applications such as tracking, 3D reconstruction and pose estimation. We develop a framework that compares PointNet features of template and source point cl…
Deep Learning-Based Point Cloud Registration: A Comprehensive Survey and Taxonomy
Southeast University（东南大学）；Purple Mountain Laboratories（紫金山实验室）；Nanyang Technological University（南洋理工大学）；Lee Kong Chian School of Medicine（李光前医学院）；Huazhong University of Science and Technology（华中科技大学）；School of Artificial Intelligence and Automation（人工智能与自动化学院）；College of Computing & Data Science（计算与数据科学学院）
University of Southern California（南加州大学）；Tencent America（腾讯美国）；Tencent Media Lab（腾讯媒体实验室）
An unsupervised point cloud object retrieval and pose estimation method, called PCRP, is proposed in this work. It is assumed that there exists a gallery point cloud set that contains point cloud objects with given pose orientation information. PCRP attempts to register the unknown point cloud object with those in the gallery set so as to achieve content-based object retrieval and pose estimation jointly, where the point cloud registration task is built upon an enhanced version of the unsupervised R-PointHop method. Experiments on the ModelNet40 dataset demonstrate the superior performance of …
GeGS-PCR: Effective and Robust 3D Point Cloud Registration with Two-Stage Color-Enhanced Geometric-3DGS Fusion
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence（人机混合增强智能国家级重点实验室）；National Engineering Research Center of Visual Information and Applications（视觉信息与应用国家工程研究中心）；Institute of Artificial Intelligence and Robotics（人工智能与机器人研究院）；Xi’an Jiaotong University（西安交通大学）
GS-Matching: Reconsidering Feature Matching task in Point Cloud Registration
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Guangdong Provincial Key Laboratory of Construction Robotics and Intelligent Construction（广东省建造机器人与智能建筑重点实验室）
DefTransNet: A Transformer-based Method for Non-Rigid Point Cloud Registration in the Simulation of Soft Tissue Deformation
Mannheim Institute for Intelligent Systems in Medicine (MIISM), Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany（曼海姆智能医学研究所（MIISM）、曼海姆医学院、海德堡大学）；Department of Radiation Oncology, Brigham and Women's Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, USA（放射肿瘤科、布里格姆和女性医院、达纳-法伯癌症研究所、哈佛医学院）；International Institute of Information Technology, Bangalore, India（国际信息技术研究所，班加罗尔，印度）；Department of Otorhinolaryngology, Head and Neck Surgery, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany（耳鼻喉科与头颈外科、曼海姆医学院、海德堡大学）；Interdisciplinary Center for Scientific Computing (IWR), Heidelberg University, Heidelberg, Germany（跨学科科学计算中心（IWR）、海德堡大学）；Central Institute for Computer Engineering (ZITI), Heidelberg University, Heidelberg, Germany（计算机工程研究中心（ZITI）、海德堡大学）；CZS Heidelberg Center for Model-Based AI, Heidelberg University, Mannheim, Germany（海德堡模型驱动人工智能中心（CZS）、海德堡大学）
Rigid Point Cloud Registration (PCR) algorithms aim to estimate the 6-DOF relative motion between two point clouds, which is important in various fields, including autonomous driving. Recent years have seen a significant improvement in global PCR algorithms, i.e. algorithms that can handle a large relative motion. This has been demonstrated in various scenarios, including indoor scenes, but has only been minimally tested in the Automotive setting, where point clouds are produced by vehicle-mounted LiDAR sensors. In this work, we aim to answer questions that are important for automotive applica…
Carnegie Mellon University（卡内基梅隆大学）；Indian Institute of Technology, Madras（马德拉斯印度理工学院）
Point-cloud registration (PCR) is an important task in various applications such as robotic manipulation, augmented and virtual reality, SLAM, etc. PCR is an optimization problem involving minimization over two different types of interdependent variables: transformation parameters and point-to-point correspondences. Recent developments in deep-learning have produced computationally fast approaches for PCR. The loss functions that are optimized in these networks are based on the error in the transformation parameters. We hypothesize that these methods would perform significantly better if they …
TurboReg: TurboClique for Robust and Efficient Point Cloud Registration
School of Remote Sensing and Information Engineering, Wuhan University（武汉大学遥感与信息工程学院）；Department of Computer and Systems Engineering, University of Zaragoza（阿拉维扎大学计算机与系统工程系）；College of Computer Science, Beijing University of Technology（北京理工大学计算机学院）；School of Computer Science, Wuhan University（武汉大学计算机学院）
RANSAC Back to SOTA: A Two-stage Consensus Filtering for Real-time 3D Registration
School of Computer Science, Wuhan University（武汉大学计算机学院）；Hong Kong Polytechnic University（香港理工大学）；School of Remote Sensing and Information Engineering, Wuhan University（武汉大学遥感信息工程学院）
针对 RANSAC 在高外点率下迭代激增的问题，本文提出 TCF，通过长度、角度和距离一致性的两阶段滤波实现实时三维点云配准，并在 KITTI、ETH 上较 MAC 最高加速三个数量级。
The University of Sydney（悉尼大学）；University of Technology Sydney（悉尼科技大学）
We present a fast feature-metric point cloud registration framework, which enforces the optimisation of registration by minimising a feature-metric projection error without correspondences. The advantage of the feature-metric projection error is robust to noise, outliers and density difference in contrast to the geometric projection error. Besides, minimising the feature-metric projection error does not need to search the correspondences so that the optimisation speed is fast. The principle behind the proposed method is that the feature difference is smallest if point clouds are aligned very w…
Point cloud registration is a fundamental problem in 3D computer vision, graphics and robotics. For the last few decades, existing registration algorithms have struggled in situations with large transformations, noise, and time constraints. In this paper, we introduce Deep Gaussian Mixture Registration (DeepGMR), the first learning-based registration method that explicitly leverages a probabilistic registration paradigm by formulating registration as the minimization of KL-divergence between two probability distributions modeled as mixtures of Gaussians. We design a neural network that extract…
Instituto Superior Técnico（高等技术学院）；Technical University of Munich（慕尼黑工业大学）；Fondazione Istituto Italiano di Tecnologia（意大利技术研究院）
We tackle data-driven 3D point cloud registration. Given point correspondences, the standard Kabsch algorithm provides an optimal rotation estimate. This allows to train registration models in an end-to-end manner by differentiating the SVD operation. However, given the initial rotation estimate supplied by Kabsch, we show we can improve point correspondence learning during model training by extending the original optimization problem. In particular, we linearize the governing constraints of the rotation matrix and solve the resulting linear system of equations. We then iteratively produce new…
University of Bonn（波恩大学）；Sapienza University of Rome（罗马大学）
The ability to build maps is a key functionality for the majority of mobile robots. A central ingredient to most mapping systems is the registration or alignment of the recorded sensor data. In this paper, we present a general methodology for photometric registration that can deal with multiple different cues. We provide examples for registering RGBD as well as 3D LIDAR data. In contrast to popular point cloud registration approaches such as ICP our method does not rely on explicit data association and exploits multiple modalities such as raw range and image data streams. Color, depth, and nor…
Concordia Institute for Information Systems Engineering（康考迪亚信息系统工程学院）；Concordia University（康考迪亚大学）
We propose a nonrigid registration approach for diffusion tensor images using a multicomponent information-theoretic measure. Explicit orientation optimization is enabled by incorporating tensor reorientation, which is necessary for wrapping diffusion tensor images. Experimental results on diffusion tensor images indicate the feasibility of the proposed approach and a much better performance compared to the affine registration method based on mutual information in terms of registration accuracy in the presence of geometric distortion.
University of Alberta（阿尔伯塔大学）；Hospital Aleman（阿勒曼医院）
In this work, we present a novel unsupervised image registration algorithm. It is differentiable end-to-end and can be used for both multi-modal and mono-modal registration. This is done using mutual information (MI) as a metric. The novelty here is that rather than using traditional ways of approximating MI, we use a neural estimator called MINE and supplement it with matrix exponential for transformation matrix computation. This leads to improved results as compared to the standard algorithms available out-of-the-box in state-of-the-art image registration toolboxes.
We present an information-theoretic approach to the registration of images with directional information, and especially for diffusion-Weighted Images (DWI), with explicit optimization over the directional scale. We call it Locally Orderless Registration with Directions (LORD). We focus on normalized mutual information as a robust information-theoretic similarity measure for DWI. The framework is an extension of the LOR-DWI density-based hierarchical scale-space model that varies and optimizes the integration, spatial, directional, and intensity scales. As affine transformations are insufficien…
University of Alberta（阿尔伯塔大学）；Hospital Aleman（阿莱曼医院）
Autograd-based software packages have recently renewed interest in image registration using homography and other geometric models by gradient descent and optimization, e.g., AirLab and DRMIME. In this work, we emphasize on using complex matrix exponential (CME) over real matrix exponential to compute transformation matrices. CME is theoretically more suitable and practically provides faster convergence as our experiments show. Further, we demonstrate that the use of an ordinary differential equation (ODE) as an optimizable dynamical system can adapt the transformation matrix more accurately to…
Probabilistic methods for point set registration have interesting theoretical properties, such as linear complexity in the number of used points, and they easily generalize to joint registration of multiple point sets. In this work, we improve their recognition performance to match state of the art. This is done by incorporating learned features, by adding a von Mises-Fisher feature model in each mixture component, and by using learned attention weights. We learn these jointly using a registration loss learning strategy (RLL) that directly uses the registration error as a loss, by back-propaga…
We propose a deformable registration algorithm based on unsupervised learning of a low-dimensional probabilistic parameterization of deformations. We model registration in a probabilistic and generative fashion, by applying a conditional variational autoencoder (CVAE) network. This model enables to also generate normal or pathological deformations of any new image based on the probabilistic latent space. Most recent learning-based registration algorithms use supervised labels or deformation models, that miss important properties such as diffeomorphism and sufficiently regular deformation field…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Dense vertex-to-vertex correspondence between 3D faces is a fundamental and challenging issue for 3D&2D face analysis. While the sparse landmarks have anatomically ground-truth correspondence, the dense vertex correspondences on most facial regions are unknown. In this view, the current literatures commonly result in reasonable but diverse solutions, which deviate from the optimum to the 3D face dense registration problem. In this paper, we revisit dense registration by a dimension-degraded problem, i.e. proportional segmentation of a line, and employ an iterative dividing and diffusing method…
This paper addresses the issue of matching rigid 3D object points with 2D image points through point registration based on maximum likelihood principle in computer simulated images. Perspective projection is necessary when transforming 3D coordinate into 2D. The problem then recasts into a missing data framework where unknown correspondences are handled via mixture models. Adopting the Expectation Conditional Maximization for Point Registration (ECMPR), two different rotation and translation optimization algorithms are compared in this paper. We analyze in detail the associated consequences in…
A Novel Streamline-based diffusion MRI Tractography Registration Method with Probabilistic Keypoint Detection
University of Electronic Science and Technology of China（电子科技大学）；Nanjing University of Science and Technology（南京理工大学）；Harvard Medical School（哈佛医学院）；Brigham and Women’s Hospital（布莱根和妇女医院）
Point base registration is an important part in many machine VISIOn applications, medical diagnostics, agricultural studies etc. The goal of point set registration is to find correspondences between different data sets and estimate the appropriate transformation that can map one set to another. Here we introduce a novel method for matching of different data sets based on Laplacian distribution. We consider the alignment of two point sets as probability density estimation problem. By using maximum likelihood methods we try to fit the Laplacian mixture model (LMM) centroids (source point set) to…
University of California, Irvine（加州大学欧文分校）；Yale University（耶鲁大学）
Diffeomorphic image registration, offering smooth transformation and topology preservation, is required in many medical image analysis tasks.Traditional methods impose certain modeling constraints on the space of admissible transformations and use optimization to find the optimal transformation between two images. Specifying the right space of admissible transformations is challenging: the registration quality can be poor if the space is too restrictive, while the optimization can be hard to solve if the space is too general. Recent learning-based methods, utilizing deep neural networks to lea…
