Building 3D geometric maps of man-made spaces is a well-established and active field that is fundamental to computer vision and robotics. However, considering the evolving nature of built environments, it is essential to question the capabilities of current mapping efforts in handling temporal changes. In addition, spatiotemporal mapping holds significant potential for achieving sustainability and circularity goals. Existing mapping approaches focus on small changes, such as object relocation or self-driving car operation; in all cases where the main structure of the scene remains fixed. Consequently, these approaches fail to address more radical changes in the structure of the built environment, such as geometry and topology. To this end, we introduce the Nothing Stands Still (NSS) benchmark, which focuses on the spatiotemporal registration of 3D scenes undergoing large spatial and temporal change, ultimately creating one coherent spatiotemporal map. Specifically, the benchmark involves registering two or more partial 3D point clouds (fragments) from the same scene but captured from different spatiotemporal views. In addition to the standard pairwise registration, we assess the multi-way registration of multiple fragments that belong to any temporal stage. As part of NSS, we introduce a dataset of 3D point clouds recurrently captured in large-scale building indoor environments that are under construction or renovation. The NSS benchmark presents three scenarios of increasing difficulty, to quantify the generalization ability of point cloud registration methods over space (within one building and across buildings) and time. We conduct extensive evaluations of state-of-the-art methods on NSS. The results demonstrate the necessity for novel methods specifically designed to handle large spatiotemporal changes. The homepage of our benchmark is at http://nothing-stands-still.com.
In this paper, we propose three novel and important methods for the registration of histological images for 3D reconstruction. First, possible intensity variations and nonstandardness in images are corrected by an intensity standardization process which maps the image scale into a standard scale where the similar intensities correspond to similar tissues meaning. Second, 2D histological images are mapped into a feature space where continuous variables are used as high confidence image features for accurate registration. Third, we propose an automatic best reference slice selection algorithm th…
Functional registration algorithms represent point clouds as functions (e.g. spacial occupancy field) avoiding unreliable correspondence estimation in conventional least-squares registration algorithms. However, existing functional registration algorithms are computationally expensive. Furthermore, the capability of registration with unknown scale is necessary in tasks such as CAD model-based object localization, yet no such support exists in functional registration. In this work, we propose a scale-invariant, linear time complexity functional registration algorithm. We achieve linear time com…
King’s College London（伦敦国王学院）；Siemens Healthineers（西门子医疗）
Registering CT images of the chest is a crucial step for several tasks such as disease progression tracking or surgical planning. It is also a challenging step because of the heterogeneous content of the human abdomen which implies complex deformations. In this work, we focus on accurately registering a subset of organs of interest. We register organ surface point clouds, as may typically be extracted from an automatic segmentation pipeline, by expanding the Bayesian Coherent Point Drift algorithm (BCPD). We introduce MO-BCPD, a multi-organ version of the BCPD algorithm which explicitly models…
Northwestern Polytechnical University（西北工业大学）；Chang’an University（长安大学）；Hubei University（湖北大学）；Huazhong University of Science and Technology（华中科技大学）
This paper focuses on developing efficient and robust evaluation metrics for RANSAC hypotheses to achieve accurate 3D rigid registration. Estimating six-degree-of-freedom (6-DoF) pose from feature correspondences remains a popular approach to 3D rigid registration, where random sample consensus (RANSAC) is a de-facto choice to this problem. However, existing metrics for RANSAC hypotheses are either time-consuming or sensitive to common nuisances, parameter variations, and different application scenarios, resulting in performance deterioration in overall registration accuracy and speed. We alle…
East China University of Science and Technology（华东理工大学）
Rotation search and point cloud registration are two fundamental problems in robotics and computer vision, which aim to estimate the rotation and the transformation between the 3D vector sets and point clouds, respectively. Due to the presence of outliers, probably in very large numbers, among the putative vector or point correspondences in real-world applications, robust estimation is of great importance. In this paper, we present ICOS (Inlier searching using COmpatible Structures), a novel, efficient and highly robust solver for both the correspondence-based rotation search and point cloud r…
Instituto Superior Técnico（高等技术学院）；Technical University of Munich（慕尼黑工业大学）；Fondazione Istituto Italiano di Tecnologia（意大利技术研究院）
We tackle data-driven 3D point cloud registration. Given point correspondences, the standard Kabsch algorithm provides an optimal rotation estimate. This allows to train registration models in an end-to-end manner by differentiating the SVD operation. However, given the initial rotation estimate supplied by Kabsch, we show we can improve point correspondence learning during model training by extending the original optimization problem. In particular, we linearize the governing constraints of the rotation matrix and solve the resulting linear system of equations. We then iteratively produce new…
Multi-view point cloud registration is a hot topic in the communities of multimedia technology and artificial intelligence (AI). In this paper, we propose a framework to reconstruct the 3D models by the multi-view point cloud registration algorithm with adaptive convergence threshold, and subsequently apply it to 3D model retrieval. The iterative closest point (ICP) algorithm is implemented combining with the motion average algorithm for the registration of multi-view point clouds. After the registration process, we design applications for 3D model retrieval. The geometric saliency map is comp…
PARTE: Plane-Assisted Robust Transformation Estimation for Point Cloud Registration
Colorado School of Mines（科罗拉多矿业大学）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；U.S. Army Combat Capabilities Development Command, Army Research Laboratory（美国陆军作战能力发展司令部陆军研究实验室）
E2PNet: Event to Point Cloud Registration with Spatio-Temporal Representation Learning
Intel Labs（英特尔实验室）；Apple Inc.（苹果公司）；Key Laboratory of Multimedia Trusted Perception and Efficient Computing, XMU（厦门大学多媒体可信感知与高效计算重点实验室）；Yancheng Institute Of Technology（盐城工学院）
In this paper, we propose a computational framework for 3D volume reconstruction from 2D histological slices using registration algorithms in feature space. To improve the quality of reconstructed 3D volume, first, intensity variations in images are corrected by an intensity standardization process which maps image intensity scale to a standard scale where similar intensities correspond to similar tissues. Second, a subvolume approach is proposed for 3D reconstruction by dividing standardized slices into groups. Third, in order to improve the quality of the reconstruction process, an automatic…
Università degli Studi di Modena e Reggio Emilia（摩德纳和雷焦艾米利亚大学）
With the spread of wearable devices and head mounted cameras, a wide range of application requiring precise user localization is now possible. In this paper we propose to treat the problem of obtaining the user position with respect to a known environment as a video registration problem. Video registration, i.e. the task of aligning an input video sequence to a pre-built 3D model, relies on a matching process of local keypoints extracted on the query sequence to a 3D point cloud. The overall registration performance is strictly tied to the actual quality of this 2D-3D matching, and can degrade…
Much progress has been made on the task of learning-based 3D point cloud registration, with existing methods yielding outstanding results on standard benchmarks, such as ModelNet40, even in the partial-to-partial matching scenario. Unfortunately, these methods still struggle in the presence of real data. In this work, we identify the sources of these failures, analyze the reasons behind them, and propose solutions to tackle them. We summarise our findings into a set of guidelines and demonstrate their effectiveness by applying them to different baseline methods, DCP and IDAM. In short, our gui…
OCA: ODE-Driven Cross-Attention for Image-to-Point-Cloud Registration
Huazhong University of Science and Technology（华中科技大学）；Northwestern Polytechnical University（西北工业大学）；Huazhong Agricultural University（华中农业大学）；Delft University of Technology（代尔夫特理工大学）
Mask 2D-3D: Adaptive Dual-Masked Autoencoder Network for Image-to-Point Cloud Registration
Hefei University of Technology（合肥工业大学）；University of Science and Technology of China（中国科学技术大学）；Institute of Advanced Technology, University of Science and Technology of China（中国科学技术大学先进技术研究院）
Hunan University（湖南大学）；Xidian University（西安电子科技大学）；The University of Western Australia（西澳大学）
In this paper, we propose a novel minimum gravitational potential energy (MPE)-based algorithm for global point set registration. The feature descriptors extraction algorithms have emerged as the standard approach to align point sets in the past few decades. However, the alignment can be challenging to take effect when the point set suffers from raw point data problems such as noises (Gaussian and Uniformly). Different from the most existing point set registration methods which usually extract the descriptors to find correspondences between point sets, our proposed MPE alignment method is able…
East China University of Science and Technology（华东理工大学）
3D point cloud registration ranks among the most fundamental problems in remote sensing, photogrammetry, robotics and geometric computer vision. Due to the limited accuracy of 3D feature matching techniques, outliers may exist, sometimes even in very large numbers, among the correspondences. Since existing robust solvers may encounter high computational cost or restricted robustness, we propose a novel, fast and highly robust solution, named VOCRA (VOting with Cost function and Rotating Averaging), for the point cloud registration problem with extreme outlier rates. Our first contribution is t…
Technical University of Munich（慕尼黑工业大学）；National University of Defense Technology（国防科技大学）；Siemens AG（西门子股份公司）
We study the problem of extracting correspondences between a pair of point clouds for registration. For correspondence retrieval, existing works benefit from matching sparse keypoints detected from dense points but usually struggle to guarantee their repeatability. To address this issue, we present CoFiNet - Coarse-to-Fine Network which extracts hierarchical correspondences from coarse to fine without keypoint detection. On a coarse scale and guided by a weighting scheme, our model firstly learns to match down-sampled nodes whose vicinity points share more overlap, which significantly shrinks …
University of Nevada, Reno（内华达大学雷诺分校）；University of Prince Edward Island（爱德华王子岛大学）；Japan Advanced Institute of Science and Technology (JAIST)（日本北陆先端科学技术大学院大学（JAIST））
This work focuses on Registration or Alignment of 3D point sets. Although the Registration problem is a well established problem and it's solved using multiple variants of Iterative Closest Point (ICP) Algorithm, most of the approaches in the current state of the art still suffers from misalignment when the \textit{Source} and the \textit{Target} point sets are separated by large rotations and translation. In this work, we propose a variant of the Standard ICP algorithm, where we introduce a Correntropy Relationship Matrix in the computation of rotation and translation component which attempts…
The University of Adelaide（阿德莱德大学）；University of Potsdam（波茨坦大学）
A popular paradigm for 3D point cloud registration is by extracting 3D keypoint correspondences, then estimating the registration function from the correspondences using a robust algorithm. However, many existing 3D keypoint techniques tend to produce large proportions of erroneous correspondences or outliers, which significantly increases the cost of robust estimation. An alternative approach is to directly search for the subset of correspondences that are pairwise consistent, without optimising the registration function. This gives rise to the combinatorial problem of matching with pairwise …
We present a new fully-automatic non-rigid 3D shape registration (morphing) framework comprising (1) a new 3D landmarking and pose normalisation method; (2) an adaptive shape template method to accelerate the convergence of registration algorithms and achieve a better final shape correspondence and (3) a new iterative registration method that combines Iterative Closest Points with Coherent Point Drift (CPD) to achieve a more stable and accurate correspondence establishment than standard CPD. We call this new morphing approach Iterative Coherent Point Drift (ICPD). Our proposed framework is eva…
Graduate School at Shenzhen, Tsinghua University（清华大学深圳研究生院）；The University of Texas at Arlington（德克萨斯大学阿灵顿分校）
Accurate registration of medical images is vital for doctor's diagnosis and quantitative analysis. In this paper, we propose a new deformable medical image registration method based on average geometric transformations and VoxelMorph CNN architecture. We compute the differential geometric information including Jacobian determinant(JD) and the curl vector(CV) of diffeomorphic registration field and use them as multi-channel of VoxelMorph CNN for second train. In addition, we use the average transformation to construct a standard brain MRI atlas which can be used as fixed image. We verify our me…
We propose a robust approach for the registration of two sets of 3D points in the presence of a large amount of outliers. Our first contribution is to reformulate the registration problem using a Truncated Least Squares (TLS) cost that makes the estimation insensitive to a large fraction of spurious point-to-point correspondences. The second contribution is a general framework to decouple rotation, translation, and scale estimation, which allows solving in cascade for the three transformations. Since each subproblem (scale, rotation, and translation estimation) is still non-convex and combinat…
This report presents a generic image registration benchmark with automatic evaluation using landmark annotations. The key features of the BIRL framework are: easily extendable, performance evaluation, parallel experimentation, simple visualisations, experiment's time-out limit, resuming unfinished experiments. From the research practice, we identified and focused on these two main use-cases: (a) comparison of user's (newly developed) method with some State-of-the-Art (SOTA) methods on a common dataset and (b) experimenting SOTA methods on user's custom dataset (which should contain landmark an…
In this paper we address the problem of establishing correspondences between different instances of the same object. The problem is posed as finding the geometric transformation that aligns a given image pair. We use a convolutional neural network (CNN) to directly regress the parameters of the transformation model. The alignment problem is defined in the setting where an unordered set of semantic key-points per image are available, but, without the correspondence information. To this end we propose a novel loss function based on cyclic consistency that solves this 2D point set registration pr…
Eliminating Registration Bias in Synthetic CT Generation: A Physics-Based Simulation Framework
organization= Medical University of Vienna, Department of Radiation Oncology , city= Vienna , country= Austria；organization= Christian Doppler Laboratory for Image；Knowledge Driven Precision Radiation Oncology, Medical University of Vienna , city= Vienna , country= Austria；organization= University of Applied Sciences Wiener Neustadt, Competence Center for Preclinical Imaging \& Biomedical Engineering , city= Wiener Neustadt , country= Austria
C3VDReg: A Benchmark for Local-to-Local Colonoscopic Registration toward Anatomical Localization
UCL Hawkes Institute（UCL哈维斯研究所）；University College London, University of London（伦敦大学学院, 伦敦大学）；Division of Informatics, Imaging and Data Sciences（信息学、成像与数据科学部门）；University of Manchester（曼彻斯特大学）
New York University（纽约大学）；NYU Abu Dhabi（纽约大学阿布扎比分校）；NYU Tandon School of Engineering（纽约大学坦登工程学院）
Given new pairs of source and target point sets, standard point set registration methods often repeatedly conduct the independent iterative search of desired geometric transformation to align the source point set with the target one. This limits their use in applications to handle the real-time point set registration with large volume dataset. This paper presents a novel method, named coherent point drift networks (CPD-Net), for the unsupervised learning of geometric transformation towards real-time non-rigid point set registration. In contrast to previous efforts (e.g. coherent point drift), …
Deformable Image Registration of Dark-Field Chest Radiographs for Local Lung Signal Change Assessment
Technical University of Munich（慕尼黑工业大学）；Munich Institute for Biomedical Engineering, Technical University of Munich（慕尼黑工业大学慕尼黑生物医学工程研究所）；Klinikum rechts der Isar, Technical University of Munich（慕尼黑工业大学右岸医院）；TUM University Hospital（慕尼黑工业大学大学医院）；Philips Innovative Technologies（飞利浦创新技术公司）；Institute for Advanced Study, Technical University of Munich（慕尼黑工业大学高等研究院）
Bruno Kessler Foundation（布鲁诺·凯斯勒基金会）；University of Trento（特伦托大学）
Diffusion magnetic resonance imaging (dMRI) and tractography provide means to study the anatomical structures within the white matter of the brain. When studying tractography data across subjects, it is usually necessary to align, i.e. to register, tractographies together. This registration step is most often performed by applying the transformation resulting from the registration of other volumetric images (T1, FA). In contrast with registration methods that "transform" tractographies, in this work, we try to find which streamline in one tractography correspond to which streamline in the othe…
