In this paper, we address the problem of registering multiple point clouds corrupted with high anisotropic localization noise. Our approach follows the widely used framework of Gaussian mixture model (GMM) reconstruction with an expectation-maximization (EM) algorithm. Existing methods are based on an implicit assumption of space-invariant isotropic Gaussian noise. However, this assumption is violated in practice in applications such as single molecule localization microscopy (SMLM). To address this issue, we propose to introduce an explicit localization noise model that decouples shape modeling with the GMM from noise handling. We design a stochastic EM algorithm that considers noise-free data as a latent variable, with closed-form solutions at each EM step. The first advantage of our approach is to handle space-variant and anisotropic Gaussian noise with arbitrary covariances. The second advantage is to leverage the explicit noise model to impose prior knowledge about the noise that may be available from physical sensors. We show on various simulated data that our noise handling strategy improves significantly the robustness to high levels of anisotropic noise. We also demonstrate the performance of our method on real SMLM data.
We present a novel, end-to-end learnable, multiview 3D point cloud registration algorithm. Registration of multiple scans typically follows a two-stage pipeline: the initial pairwise alignment and the globally consistent refinement. The former is often ambiguous due to the low overlap of neighboring point clouds, symmetries and repetitive scene parts. Therefore, the latter global refinement aims at establishing the cyclic consistency across multiple scans and helps in resolving the ambiguous cases. In this paper we propose, to the best of our knowledge, the first end-to-end algorithm for joint…
Correspondence-Free Multiview Point Cloud Registration via Depth-Guided Joint Optimisation
Robotics Institute, Faculty of Engineering and Information Technology, University of Technology Sydney（机器人研究所，工程与信息技术学院，新南威尔士大学）；School of Informatics, The University of Edinburgh（信息学院，爱丁堡大学）
Multiview Point Cloud Registration Based on Minimum Potential Energy for Free-Form Blade Measurement
College of Electrical and Information Engineering, Hunan University（湖南大学电气与信息工程学院）；School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；Department of Computer Science and Software Engineering, The University of Western Australia（西澳大学计算机科学与软件工程系）
Video provides us with the spatio-temporal consistency needed for visual learning. Recent approaches have utilized this signal to learn correspondence estimation from close-by frame pairs. However, by only relying on close-by frame pairs, those approaches miss out on the richer long-range consistency between distant overlapping frames. To address this, we propose a self-supervised approach for correspondence estimation that learns from multiview consistency in short RGB-D video sequences. Our approach combines pairwise correspondence estimation and registration with a novel SE(3) transformatio…
MemReg: Streaming Outdoor LiDAR Point Cloud Registration with Hybrid Memory Buffers
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University（上海交通大学自动化与智能感知学院）；University of Twente（特文特大学）；University of Cambridge（剑桥大学）；University of Bath（巴斯大学）；Phigent Robotics（飞劲机器人公司）
University of Pennsylvania（宾夕法尼亚大学）；University of Minnesota（明尼苏达大学）
This paper presents a semi-supervised learning framework to train a keypoint detector using multiview image streams given the limited labeled data (typically $<$4\%). We leverage the complementary relationship between multiview geometry and visual tracking to provide three types of supervisionary signals to utilize the unlabeled data: (1) keypoint detection in one view can be supervised by other views via the epipolar geometry; (2) a keypoint moves smoothly over time where its optical flow can be used to temporally supervise consecutive image frames to each other; (3) visible keypoint in one v…
University of Rochester（罗切斯特大学）；University of Maryland, College Park（马里兰大学帕克分校）；Rutgers University（罗格斯大学）；Z2W Corporation（Z2W公司）；Chinese Academy of Sciences（中国科学院）；Peng Cheng Laboratory（鹏城实验室）
We propose to tackle the problem of multiview 2D/3D rigid registration for intervention via a Point-Of-Interest Network for Tracking and Triangulation ($\text{POINT}^2$). $\text{POINT}^2$ learns to establish 2D point-to-point correspondences between the pre- and intra-intervention images by tracking a set of random POIs. The 3D pose of the pre-intervention volume is then estimated through a triangulation layer. In $\text{POINT}^2$, the unified framework of the POI tracker and the triangulation layer enables learning informative 2D features and estimating 3D pose jointly. In contrast to existin…
National University of Singapore（新加坡国立大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Probabilistic point cloud registration methods are becoming more popular because of their robustness. However, unlike point-to-plane variants of iterative closest point (ICP) which incorporate local surface geometric information such as surface normals, most probabilistic methods (e.g., coherent point drift (CPD)) ignore such information and build Gaussian mixture models (GMMs) with isotropic Gaussian covariances. This results in sphere-like GMM components which only penalize the point-to-point distance between the two point clouds. In this paper, we propose a novel method called CPD with Loca…
Point cloud registration sits at the core of many important and challenging 3D perception problems including autonomous navigation, SLAM, object/scene recognition, and augmented reality. In this paper, we present a new registration algorithm that is able to achieve state-of-the-art speed and accuracy through its use of a hierarchical Gaussian Mixture Model (GMM) representation. Our method constructs a top-down multi-scale representation of point cloud data by recursively running many small-scale data likelihood segmentations in parallel on a GPU. We leverage the resulting representation using …
Does DINOv3 Set a New Medical Vision Standard? Benchmarking 2D and 3D Classification, Segmentation, and Registration
Imperial College London（伦敦帝国理工学院）；University of Science and Technology of China（中国科学技术大学）；Dresden University of Technology（德累斯顿技术大学）；University of Erlangen-Nuremberg（埃尔兰根-纽伦堡大学）；University of Oxford（牛津大学）；University of Sheffield（谢菲尔德大学）；Technical University of Munich (TUM)（慕尼黑技术大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；The Hong Kong University of Science and Technology（香港科学与技术大学）；The Chinese University of Hong Kong（香港中文大学）；Yale University（耶鲁大学）
University of California, Riverside（加州大学河滨分校）；Indian Institute of Science（印度科学学院）
Consider the problem of registering multiple point sets in some $d$-dimensional space using rotations and translations. Assume that there are sets with common points, and moreover the pairwise correspondences are known for such sets. We consider a least-squares formulation of this problem, where the variables are the transforms associated with the point sets. The present novelty is that we reduce this nonconvex problem to an optimization over the positive semidefinite cone, where the objective is linear but the constraints are nevertheless nonconvex. We propose to solve this using variable spl…
Faster 3D Gaussian Splatting Convergence via Structure-Aware Densification
Cambridge University（剑桥大学）；Saarbrücken Research Center for Visual Computing, Interaction, and Artificial Intelligence (VIA)（萨尔布吕肯视觉计算、交互与人工智能研究中心（VIA））
Analysis of microscopy images can provide insight into many biological processes. One particularly challenging problem is cell nuclear segmentation in highly anisotropic and noisy 3D image data. Manually localizing and segmenting each and every cell nuclei is very time consuming, which remains a bottleneck in large scale biological experiments. In this work we present a tool for automated segmentation of cell nuclei from 3D fluorescent microscopic data. Our tool is based on state-of-the-art image processing and machine learning techniques and supports a friendly graphical user interface (GUI).…
Facebook AI Research（脸书人工智能研究院）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
Pretraining on large labeled datasets is a prerequisite to achieve good performance in many computer vision tasks like 2D object recognition, video classification etc. However, pretraining is not widely used for 3D recognition tasks where state-of-the-art methods train models from scratch. A primary reason is the lack of large annotated datasets because 3D data is both difficult to acquire and time consuming to label. We present a simple self-supervised pertaining method that can work with any 3D data - single or multiview, indoor or outdoor, acquired by varied sensors, without 3D registration…
