Consider the problem of registering multiple point sets in some $d$-dimensional space using rotations and translations. Assume that there are sets with common points, and moreover the pairwise correspondences are known for such sets. We consider a least-squares formulation of this problem, where the variables are the transforms associated with the point sets. The present novelty is that we reduce this nonconvex problem to an optimization over the positive semidefinite cone, where the objective is linear but the constraints are nevertheless nonconvex. We propose to solve this using variable splitting and the alternating directions method of multipliers (ADMM). Due to the linearity of the objective and the structure of constraints, the ADMM subproblems are given by projections with closed-form solutions. In particular, for $m$ point sets, the dominant cost per iteration is the partial eigendecomposition of an $md \times md$ matrix, and $m-1$ singular value decompositions of $d \times d$ matrices. We empirically show that for appropriate parameter settings, the proposed solver has a large convergence basin and is stable under perturbations. As applications, we use our method for $2$D shape matching and $3$D multiview registration. In either application, we model the shapes/scans as point sets and determine the pairwise correspondences using ICP. In particular, our algorithm compares favorably with existing methods for multiview reconstruction in terms of timing and accuracy.
The University of Sydney（悉尼大学）；University of Technology Sydney（悉尼科技大学）
We present a fast feature-metric point cloud registration framework, which enforces the optimisation of registration by minimising a feature-metric projection error without correspondences. The advantage of the feature-metric projection error is robust to noise, outliers and density difference in contrast to the geometric projection error. Besides, minimising the feature-metric projection error does not need to search the correspondences so that the optimisation speed is fast. The principle behind the proposed method is that the feature difference is smallest if point clouds are aligned very w…
University of Texas, Austin（德克萨斯大学奥斯汀分校）；University of California, Berkeley（加州大学伯克利分校）
Motivated by an emerging theory of robust low-rank matrix representation, in this paper, we introduce a novel solution for online rigid-body motion registration. The goal is to develop algorithmic techniques that enable a robust, real-time motion registration solution suitable for low-cost, portable 3-D camera devices. Assuming 3-D image features are tracked via a standard tracker, the algorithm first utilizes Robust PCA to initialize a low-rank shape representation of the rigid body. Robust PCA finds the global optimal solution of the initialization, while its complexity is comparable to sing…
College of Science, Beijing Forestry University（北京林业大学理学院）；Academy of Opto-Electronics（光电研究院）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）
Point-cloud data acquired using a terrestrial laser scanner (TLS) play an important role in digital forestry research. Multiple scans are generally used to overcome occlusion effects and obtain complete tree structural information. However, it is time-consuming and difficult to place artificial reflectors in a forest with complex terrain for marker-based registration, a process that reduces registration automation and efficiency. In this study, we propose an automatic coarse-to-fine method for the registration of point-cloud data from multiple scans of a single tree. In coarse registration, po…
This paper addresses the issue of matching rigid 3D object points with 2D image points through point registration based on maximum likelihood principle in computer simulated images. Perspective projection is necessary when transforming 3D coordinate into 2D. The problem then recasts into a missing data framework where unknown correspondences are handled via mixture models. Adopting the Expectation Conditional Maximization for Point Registration (ECMPR), two different rotation and translation optimization algorithms are compared in this paper. We analyze in detail the associated consequences in…
APCReg: Anatomical-Prior-Guided Coarse-to-Fine CBCT--IOS Registration via Multi-View Projection and Reliability-Controlled Residual Correction
Hangzhou Dianzi University（杭州电子科技大学）；Shanghai Ninth People’s Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属第九人民医院）；Sir Run Run Shaw Hospital, Zhejiang University School of Medicine（浙江大学医学院附属邵逸夫医院）；Shanghai Stomatological Hospital & School of Stomatology, Fudan University（复旦大学附属口腔医院·口腔医学院）
Xi'an Jiaotong University（西安交通大学）；Jiangnan University（江南大学）；Kyushu Institute of Technology（九州工业大学）
This paper proposes an effective approach for the scaling registration of $m$-D point sets. Different from the rigid transformation, the scaling registration can not be formulated into the common least square function due to the ill-posed problem caused by the scale factor. Therefore, this paper designs a novel objective function for the scaling registration problem. The appearance of this objective function is a rational fraction, where the numerator item is the least square error and the denominator item is the square of the scale factor. By imposing the emphasis on scale factor, the ill-pos…
Purpose: The purpose of this paper is to present a method for real-time 2D-3D non-rigid registration using a single fluoroscopic image. Such a method can find applications in surgery, interventional radiology and radiotherapy. By estimating a three-dimensional displacement field from a 2D X-ray image, anatomical structures segmented in the preoperative scan can be projected onto the 2D image, thus providing a mixed reality view. Methods: A dataset composed of displacement fields and 2D projections of the anatomy is generated from the preoperative scan. From this dataset, a neural network is tr…
Functional registration algorithms represent point clouds as functions (e.g. spacial occupancy field) avoiding unreliable correspondence estimation in conventional least-squares registration algorithms. However, existing functional registration algorithms are computationally expensive. Furthermore, the capability of registration with unknown scale is necessary in tasks such as CAD model-based object localization, yet no such support exists in functional registration. In this work, we propose a scale-invariant, linear time complexity functional registration algorithm. We achieve linear time com…
Bytedance（字节跳动）；National University of Singapore（新加坡国立大学）
This paper presents DeepI2P: a novel approach for cross-modality registration between an image and a point cloud. Given an image (e.g. from a rgb-camera) and a general point cloud (e.g. from a 3D Lidar scanner) captured at different locations in the same scene, our method estimates the relative rigid transformation between the coordinate frames of the camera and Lidar. Learning common feature descriptors to establish correspondences for the registration is inherently challenging due to the lack of appearance and geometric correlations across the two modalities. We circumvent the difficulty by …
Graduate School of Informatics, Kyoto University（京都大学信息学研究科）；Graduate School of Medicine, Kyoto University（京都大学医学研究科）
Organ shape reconstruction based on a single-projection image during treatment has wide clinical scope, e.g., in image-guided radiotherapy and surgical guidance. We propose an image-to-graph convolutional network that achieves deformable registration of a 3D organ mesh for a single-viewpoint 2D projection image. This framework enables simultaneous training of two types of transformation: from the 2D projection image to a displacement map, and from the sampled per-vertex feature to a 3D displacement that satisfies the geometrical constraint of the mesh structure. Assuming application to radiati…
Friedrich-Alexander Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Siemens Healthineers AG（西门子医疗股份公司）；Graduate School in Advanced Optical Technologies (SAOT)（先进光学技术研究生院（SAOT））
Registration of pre-operative 3-D volumes to intra-operative 2-D X-ray images is important in minimally invasive medical procedures. Rigid registration can be performed by estimating a global rigid motion that optimizes the alignment of local correspondences. However, inaccurate correspondences challenge the registration performance. To minimize their influence, we estimate optimal weights for correspondences using PointNet. We train the network directly with the criterion to minimize the registration error. We propose an objective function which includes point-to-plane correspondence-based mo…
Yale University（耶鲁大学）；University of Pennsylvania（宾夕法尼亚大学）
Augmented reality has the potential to improve operating room workflow by allowing physicians to "see" inside a patient through the projection of imaging directly onto the surgical field. For this to be useful the acquired imaging must be quickly and accurately registered with patient and the registration must be maintained. Here we describe a method for projecting a CT scan with Microsoft Hololens and then aligning that projection to a set of fiduciary markers. Radio-opaque stickers with unique QR-codes are placed on an object prior to acquiring a CT scan. The location of the markers in the C…
Projection-Domain Sensitivity Analysis of Vertebral DRRs Under Intrinsic Calibration Perturbation
Department of Mathematics, Tulane University（特伦特大学数学系）；Department of Computer Science, University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校计算机科学系）；EOS, EOS Imaging Inc.（EOS公司）；Department of Electrical and Computer Engineering, Vanderbilt University（范德比大学电气与计算机工程系）；Department of Computer Science, Vanderbilt University（范德比大学计算机科学系）
University of Utah（犹他大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）
Popular 3D scan registration projects, such as Stanford digital Michelangelo or KinectFusion, exploit the high-resolution sensor data for scan alignment. It is particularly challenging to solve the registration of sparse 3D scans in the absence of RGB components. In this case, we can not establish point correspondences since the same 3D point cannot be captured in two successive scans. In contrast to correspondence based methods, we take a different viewpoint and formulate the sparse 3D registration problem based on the constraints from the intersection of line segments from adjacent scans. We…
Institute for Applied Life Sciences, University of Massachusetts（马萨诸塞大学应用生命科学研究所）；University of Massachusetts（马萨诸塞大学）
Advancement of imaging techniques enables consecutive image sequences to be acquired for quality monitoring of manufacturing production lines. Registration for these image sequences is essential for in-line pattern inspection and metrology, e.g., in the printing process of flexible electronics. However, conventional image registration algorithms cannot produce accurate results when the images contain many similar and deformable patterns in the manufacturing process. Such a failure originates from a fact that the conventional algorithms only use the spatial and pixel intensity information for r…
An unsupervised, iterative point-set registration algorithm for an unlabeled (i.e. correspondence between points is unknown) N-dimensional Euclidean point-cloud is proposed. It is based on linear least squares, and considers all possible point pairings and iteratively aligns the two sets until the number of point pairs does not exceed the maximum number of allowable one-to-one pairings.
We describe a convex programming framework for pose estimation in 2D/3D point-set registration with unknown point correspondences. We give two mixed-integer nonlinear program (MINP) formulations of the 2D/3D registration problem when there are multiple 2D images, and propose convex relaxations for both of the MINPs to semidefinite programs (SDP) that can be solved efficiently by interior point methods. Our approach to the 2D/3D registration problem is non-iterative in nature as we jointly solve for pose and correspondence. Furthermore, these convex programs can readily incorporate feature desc…
UNC Chapel Hill/BRIC（北卡罗来纳大学教堂山分校/生物医学影像中心）；Microsoft Research Cambridge（微软研究院剑桥）；University of Delaware（特拉华大学）
Correlative microscopy is a methodology combining the functionality of light microscopy with the high resolution of electron microscopy and other microscopy technologies. Image registration for correlative microscopy is quite challenging because it is a multi-modal, multi-scale and multi-dimensional registration problem. In this report, I introduce two methods of image registration for correlative microscopy. The first method is based on fiducials (beads). I generate landmarks from the fiducials and compute the similarity transformation matrix based on three pairs of nearest corresponding land…
Bio-Key International Inc.（Bio-Key国际公司）；Capella University（卡佩拉大学）
A novel minutia-based fingerprint matching algorithm is proposed that employs iterative global alignment on two minutia sets. The matcher considers all possible minutia pairings and iteratively aligns the two sets until the number of minutia pairs does not exceed the maximum number of allowable one-to-one pairings. The optimal alignment parameters are derived analytically via linear least squares. The first alignment establishes a region of overlap between the two minutia sets, which is then (iteratively) refined by each successive alignment. After each alignment, minutia pairs that exhibit we…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Dense vertex-to-vertex correspondence between 3D faces is a fundamental and challenging issue for 3D&2D face analysis. While the sparse landmarks have anatomically ground-truth correspondence, the dense vertex correspondences on most facial regions are unknown. In this view, the current literatures commonly result in reasonable but diverse solutions, which deviate from the optimum to the 3D face dense registration problem. In this paper, we revisit dense registration by a dimension-degraded problem, i.e. proportional segmentation of a line, and employ an iterative dividing and diffusing method…
PointRegGPT: Boosting 3D Point Cloud Registration using Generative Point-Cloud Pairs for Training
University of Electronic Science and Technology of China（电子科技大学）；The Chinese University of Hong Kong（香港中文大学）；Hong Kong University of Science and Technology（香港科技大学）；Megvii Technology（商汤科技）
We propose a robust approach for the registration of two sets of 3D points in the presence of a large amount of outliers. Our first contribution is to reformulate the registration problem using a Truncated Least Squares (TLS) cost that makes the estimation insensitive to a large fraction of spurious point-to-point correspondences. The second contribution is a general framework to decouple rotation, translation, and scale estimation, which allows solving in cascade for the three transformations. Since each subproblem (scale, rotation, and translation estimation) is still non-convex and combinat…
Hg-I2P: Bridging Modalities for Generalizable Image-to-Point-Cloud Registration via Heterogeneous Graphs
Huazhong University of Science and Technology（华中科技大学）；Northwestern Polytechnical University（西北工业大学）；Huazhong Agricultural University（华中农业大学）；Delft University of Technology（代尔夫特理工大学）
School of Geospatial Engineering and Science, Sun Yat-Sen University（中山大学测绘科学与技术学院）；Department of Land Surveying and Geo-Informatics, The Hong Kong Polytechnic University（香港理工大学土地测量及地理资讯学系）；Changjiang River Scientific Research Institute (CRSRI)（长江水利委员会长江科学院）；Key Laboratory of Digital Earth Science, Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院数字地球重点实验室）；Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai)（南方海洋科学与工程广东省实验室（珠海））
Registration of unmanned aerial vehicle laser scanning (ULS) and ground light detection and ranging (LiDAR) point clouds in forests is critical to create a detailed representation of a forest structure and an accurate inversion of forest parameters. However, forest occlusion poses challenges for marker-based registration methods, and some marker-free automated registration methods have low efficiency due to the process of object (e.g., tree, crown) segmentation. Therefore, we use a divide-and-conquer strategy and propose an automated and efficient method to register ULS and ground LiDAR point …
Diff-Reg v2: Diffusion-Based Matching Matrix Estimation for Image Matching and 3D Registration
PCA Lab, Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, School of Computer Science and Engineering, Nanjing University of Science and Technology（PCA实验室、教育部高维信息智能感知与系统重点实验室、计算机科学与工程学院、南京理工大学）；Nanyang Technological University（南洋理工大学）；Visual Recognition Group, Faculty of Electrical Engineering, Czech Technical University in Prague（视觉识别组、电气工程学院、布拉格捷克技术大学）；School of Artificial Intelligence and Computer Science, Jiangnan University（人工智能与计算机科学学院、江南大学）
We propose the residual expansion (RE) algorithm: a global (or near-global) optimization method for nonconvex least squares problems. Unlike most existing nonconvex optimization techniques, the RE algorithm is not based on either stochastic or multi-point searches; therefore, it can achieve fast global optimization. Moreover, the RE algorithm is easy to implement and successful in high-dimensional optimization. The RE algorithm exhibits excellent empirical performance in terms of k-means clustering, point-set registration, optimized product quantization, and blind image deblurring.
German Cancer Research Center（德国癌症研究中心）；Heidelberg University（海德堡大学）
Image registration is the basis for many applications in the fields of medical image computing and computer assisted interventions. One example is the registration of 2D X-ray images with preoperative three-dimensional computed tomography (CT) images in intraoperative surgical guidance systems. Due to the high safety requirements in medical applications, estimating registration uncertainty is of a crucial importance in such a scenario. However, previously proposed methods, including classical iterative registration methods and deep learning-based methods have one characteristic in common: They…
University of Utah（犹他大学）；Trimble（天宝公司）；Instituto Superior Técnico, University of Lisboa（里斯本大学高等理工学院）
We propose a novel technique to register sparse 3D scans in the absence of texture. While existing methods such as KinectFusion or Iterative Closest Points (ICP) heavily rely on dense point clouds, this task is particularly challenging under sparse conditions without RGB data. Sparse texture-less data does not come with high-quality boundary signal, and this prohibits the use of correspondences from corners, junctions, or boundary lines. Moreover, in the case of sparse data, it is incorrect to assume that the same point will be captured in two consecutive scans. We take a different approach an…
EdgeRegNet: Edge Feature-based Multimodal Registration Network between Images and LiDAR Point Clouds
School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；School of Software, Shandong University（山东大学软件学院）；School of Engineering and Sustainable Development, De Montfort University（德蒙福特大学工程与可持续发展学院）
As an efficient algorithm to solve the multi-view registration problem,the motion averaging (MA) algorithm has been extensively studied and many MA-based algorithms have been introduced. They aim at recovering global motions from relative motions and exploiting information redundancy to average accumulative errors. However, one property of these methods is that they use Guass-Newton method to solve a least squares problem for the increment of global motions, which may lead to low efficiency and poor robustness to outliers. In this paper, we propose a novel motion averaging framework for the mu…
Spinal ligaments detection on vertebrae meshes using registration and 3D edge detection
University of Koblenz（科布伦茨大学）；Institute for Computational Visualistics（计算视觉研究所）；Intelligent Autonomous Systems Group（智能自主系统组）；Institute for Medical Technology and Information Processing（医学技术与信息处理研究所）
The detailed reconstruction of neural anatomy for connectomics studies requires a combination of resolution and large three-dimensional data capture provided by serial section electron microscopy (ssEM). The convergence of high throughput ssEM imaging and improved tissue preparation methods now allows ssEM capture of complete specimen volumes up to cubic millimeter scale. The resulting multi-terabyte image sets span thousands of serial sections and must be precisely registered into coherent volumetric forms in which neural circuits can be traced and segmented. This paper introduces a Signal Wh…
East China University of Science and Technology（华东理工大学）
In this paper, we present IRON (Invariant-based global Robust estimation and OptimizatioN), a non-minimal and highly robust solution for point cloud registration with a great number of outliers among the correspondences. To realize this, we decouple the registration problem into the estimation of scale, rotation and translation, respectively. Our first contribution is to propose RANSIC (RANdom Samples with Invariant Compatibility), which employs the invariant compatibility to seek inliers from random samples and robustly estimates the scale between two sets of point clouds in the meantime. Onc…
University of Science and Technology of China（中国科学技术大学）；Cardiff University（卡迪夫大学）；Zhejiang University（浙江大学）
Imperfect data (noise, outliers and partial overlap) and high degrees of freedom make non-rigid registration a classical challenging problem in computer vision. Existing methods typically adopt the $\ell_{p}$ type robust estimator to regularize the fitting and smoothness, and the proximal operator is used to solve the resulting non-smooth problem. However, the slow convergence of these algorithms limits its wide applications. In this paper, we propose a formulation for robust non-rigid registration based on a globally smooth robust estimator for data fitting and regularization, which can handl…
University of Texas at Arlington（德克萨斯大学阿灵顿分校）；IBM T. J. Watson Research Center（IBM托马斯·J·沃森研究中心）
In this paper, we propose a novel method for image fusion with a high-resolution panchromatic image and a low-resolution multispectral image at the same geographical location. The fusion is formulated as a convex optimization problem which minimizes a linear combination of a least-squares fitting term and a dynamic gradient sparsity regularizer. The former is to preserve accurate spectral information of the multispectral image, while the latter is to keep sharp edges of the high-resolution panchromatic image. We further propose to simultaneously register the two images during the fusing proces…
Massachusetts Institute of Technology（麻省理工学院）；Ford Motor Company（福特汽车公司）
Using geometric landmarks like lines and planes can increase navigation accuracy and decrease map storage requirements compared to commonly-used LiDAR point cloud maps. However, landmark-based registration for applications like loop closure detection is challenging because a reliable initial guess is not available. Global landmark matching has been investigated in the literature, but these methods typically use ad hoc representations of 3D line and plane landmarks that are not invariant to large viewpoint changes, resulting in incorrect matches and high registration error. To address this issu…
Jiangnan University（江南大学）；University of Surrey（萨里大学）
Only learning one projection matrix from original samples to the corresponding binary labels is too strict and will consequentlly lose some intrinsic geometric structures of data. In this paper, we propose a novel transition subspace learning based least squares regression (TSL-LSR) model for multicategory image classification. The main idea of TSL-LSR is to learn a transition subspace between the original samples and binary labels to alleviate the problem of overfitting caused by strict projection learning. Moreover, in order to reflect the underlying low-rank structure of transition matrix a…
The present paper reviews the classical problem of free-form curve registration and applies it to an efficient RGBD visual odometry system called Canny-VO, as it efficiently tracks all Canny edge features extracted from the images. Two replacements for the distance transformation commonly used in edge registration are proposed: Approximate Nearest Neighbour Fields and Oriented Nearest Neighbour Fields. 3D2D edge alignment benefits from these alternative formulations in terms of both efficiency and accuracy. It removes the need for the more computationally demanding paradigms of datato-model re…
