Simultaneous Multi-View Camera Pose Estimation and Object Tracking with Square Planar Markers
Instituto Maimónides de Investigación Biomédica de Córdoba (IMIBIC)（科尔多瓦迈蒙尼德生物医学研究所）；Universidad de Córdoba（科尔多瓦大学）
Hamid Sarmadi, Rafael Muñoz-Salinas, M. A. Berbís, R. Medina-Carnicer
备注：
Some errors in the IEEE Access version (regarding object's rotational accuracy, and the definition of Equation 14) have been corrected in this version. IEEE Access paper: https://doi.org/10.1109/ACCESS.2019.2896648
Object tracking is a key aspect in many applications such as augmented reality in medicine (e.g. tracking a surgical instrument) or robotics. Squared planar markers have become popular tools for tracking since their pose can be estimated from their four corners. While using a single marker and a single camera limits the working area considerably, using multiple markers attached to an object requires estimating their relative position, which is not trivial, for high accuracy tracking. Likewise, using multiple cameras requires estimating their extrinsic parameters, also a tedious process that must be repeated whenever a camera is moved.
This work proposes a novel method to simultaneously solve the above-mentioned problems. From a video sequence showing a rigid set of planar markers recorded from multiple cameras, the proposed method is able to automatically obtain the three-dimensional configuration of the markers, the extrinsic parameters of the cameras, and the relative pose between the markers and the cameras at each frame. Our experiments show that our approach can obtain highly accurate results for estimating these parameters using low resolution cameras.
Once the parameters are obtained, tracking of the object can be done in real time with a low computational cost. The proposed method is a step forward in the development of cost-effective solutions for object tracking.
Instituto Maimónides de Investigación en Biomedicina (IMIBIC)（迈蒙尼德生物医学研究所）；Córdoba University（科尔多瓦大学）；Health Time；Clínica las Nieves（拉斯涅韦斯诊所）
BACKGROUND AND OBJECTIVE: Patient positioning is a crucial step in radiation therapy, for which non-invasive methods have been developed based on surface reconstruction using optical 3D imaging. However, most solutions need expensive specialized hardware and a careful calibration procedure that must be repeated over time.This paper proposes a fast and cheap patient positioning method based on inexpensive consumer level RGB-D sensors. METHODS: The proposed method relies on a 3D reconstruction approach that fuses, in real-time, artificial and natural visual landmarks recorded from a hand-held RG…
Squared planar markers are a popular tool for fast, accurate and robust camera localization, but its use is frequently limited to a single marker, or at most, to a small set of them for which their relative pose is known beforehand. Mapping and localization from a large set of planar markers is yet a scarcely treated problem in favour of keypoint-based approaches. However, while keypoint detectors are not robust to rapid motion, large changes in viewpoint, or significant changes in appearance, fiducial markers can be robustly detected under a wider range of conditions. This paper proposes a no…
Despite the attention marker-less pose estimation has attracted in recent years, marker-based approaches still provide unbeatable accuracy under controlled environmental conditions. Thus, they are used in many fields such as robotics or biomedical applications but are primarily implemented through classical approaches, which require lots of heuristics and parameter tuning for reliable performance under different environments. In this work, we propose MarkerPose, a robust, real-time pose estimation system based on a planar target of three circles and a stereo vision system. MarkerPose is meant …
Córdoba University（科尔多瓦大学）；Instituto Maimónides de Investigación en Biomedicina (IMIBIC)（迈蒙尼德生物医学研究所（IMIBIC））
This paper proposes a novel approach for Simultaneous Localization and Mapping by fusing natural and artificial landmarks. Most of the SLAM approaches use natural landmarks (such as keypoints). However, they are unstable over time, repetitive in many cases or insufficient for a robust tracking (e.g. in indoor buildings). On the other hand, other approaches have employed artificial landmarks (such as squared fiducial markers) placed in the environment to help tracking and relocalization. We propose a method that integrates both approaches in order to achieve long-term robust tracking in many sc…
Image-based fiducial markers are useful in problems such as object tracking in cluttered or textureless environments, camera (and multi-sensor) calibration tasks, and vision-based simultaneous localization and mapping (SLAM). The state-of-the-art fiducial marker detection algorithms rely on the consistency of the ambient lighting. This paper introduces LiDARTag, a novel fiducial tag design and detection algorithm suitable for light detection and ranging (LiDAR) point clouds. The proposed method runs in real-time and can process data at 100 Hz, which is faster than the currently available LiDAR…
Plane model extraction from three-dimensional point clouds is a necessary step in many different applications such as planar object reconstruction, indoor mapping and indoor localization. Different RANdom SAmple Consensus (RANSAC)-based methods have been proposed for this purpose in recent years. In this study, we propose a novel method-based on RANSAC called Multiplane Model Estimation, which can estimate multiple plane models simultaneously from a noisy point cloud using the knowledge extracted from a scene (or an object) in order to reconstruct it accurately. This method comprises two steps…
The University of Adelaide（阿德莱德大学）；University of Tsukuba（筑波大学）
Planar markers are useful in robotics and computer vision for mapping and localisation. Given a detected marker in an image, a frequent task is to estimate the 6DOF pose of the marker relative to the camera, which is an instance of planar pose estimation (PPE). Although there are mature techniques, PPE suffers from a fundamental ambiguity problem, in that there can be more than one plausible pose solutions for a PPE instance. Especially when localisation of the marker corners is noisy, it is often difficult to disambiguate the pose solutions based on reprojection error alone. Previous methods …
Asynchronous Event Stream Noise Filtering for High-frequency Structure Deformation Measurement
College of Aerospace Science and Engineering, National University of Defense Technology（航空航天科学与工程学院，国防科技大学）；Hunan Provincial Key Laboratory of Image Measurement and Vision Navigation（图像测量与视觉导航省级重点实验室）
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Camera pose tracking attracts much interest both from academic and industrial communities, of which the methods based on planar markers are easy to be implemented. However, most of the existing methods need to identify multiple points in the marker images for matching to space points. Then, PnP and RANSAC are used to compute the camera pose. If cameras move fast or are far away from markers, matching is easy to generate errors and even RANSAC cannot remove incorrect matching. Then, the result by PnP cannot have good performance. To solve this problem, we design circular markers and represent 6…
Image Sciences Institute, University Medical Center Utrecht（乌得勒支大学医学中心影像科学研究所）；University Medical Center Utrecht（乌得勒支大学医学中心）；Utrecht（乌得勒支（城市名，此处不单独作为机构，仅保留原英文机构名中的城市信息但按要求不提取城市））
Purpose: This study demonstrates a proof of concept of a method for simultaneous anatomical imaging and real-time (SMART) passive device tracking for MR-guided interventions. Methods: Phase Correlation template matching was combined with a fast undersampled radial multi-echo acquisition using the white marker phenomenon after the first echo. In this way, the first echo provides anatomical contrast, whereas the other echoes provide white marker contrast to allow accurate device localization using fast simulations and template matching. This approach was tested on tracking of five 0.5 mm steel m…
In this paper, we investigate the influence of the spatial configuration of a number of $n \geq 4$ control points on the accuracy and robustness of space resection methods, e.g. used by a fiducial marker for pose estimation. We find robust configurations of control points by minimizing the first order perturbed solution of the DLT algorithm which is equivalent to minimizing the condition number of the data matrix. An empirical statistical evaluation is presented verifying that these optimized control point configurations not only increase the performance of the DLT homography estimation but al…
University of Pennsylvania School of Engineering and Applied Science（宾夕法尼亚大学工程与应用科学学院）
TagSLAM provides a convenient, flexible, and robust way of performing Simultaneous Localization and Mapping (SLAM) with AprilTag fiducial markers. By leveraging a few simple abstractions (bodies, tags, cameras), TagSLAM provides a front end to the GTSAM factor graph optimizer that makes it possible to rapidly design a range of experiments that are based on tags: full SLAM, extrinsic camera calibration with non-overlapping views, visual localization for ground truth, loop closure for odometry, pose estimation etc. We discuss in detail how TagSLAM initializes the factor graph in a robust way, an…
Circular markers are planar markers which offer great performances for detection and pose estimation. For an uncalibrated camera with an unknown focal length, at least the images of at least two coplanar circles are generally required to recover their poses. Unfortunately, detecting more than one ellipse in the image must be tricky and time-consuming, especially regarding concentric circles. On the other hand, when the camera is calibrated, one circle suffices but the solution is twofold and can hardly be disambiguated. Our contribution is to put beyond this limit by dealing with the uncalibra…
Simultaneous hyperkinetic movement disorders phenotyping: a cross-cohort pediatric transfer study using routine videos, markerless pose estimation and a tabular foundation model
Lausanne University Hospital (CHUV)（洛桑大学医院）；University of Lausanne (UNIL)（洛桑大学）；Institut du Neurone（神经元研究所）；Clinique Beau Soleil（博索莱伊诊所）；Institut Mutualiste Montpelliérain（蒙彼利埃互助研究所）；Military University Hospital of Sfax（斯法克斯军事大学医院）；University of Edinburgh（爱丁堡大学）；Hospital Sant Joan de Déu（圣琼德迪乌医院）；European Reference Network for Rare Neurological Diseases (ERN-RND)（欧洲罕见神经系统疾病参考网络）；Instituto de Salud Carlos III（卡洛斯三世健康研究所）；CHU Montpellier（蒙彼利埃大学医院）；Umeå University（于默奥大学）；University Hospital Lausanne（洛桑大学医院）；Ecole Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；British Columbia Children’s Hospital（不列颠哥伦比亚儿童医院）
Centrum Wiskunde & Informatica（荷兰国家数学与计算机科学研究中心）；Leiden University（莱顿大学）；Thermo Fisher Scientific（赛默飞世尔科技）；Leiden Institute of Advanced Computer Science（莱顿高级计算机科学研究院）
Tilt-series alignment is crucial to obtaining high-resolution reconstructions in cryo-electron tomography. Beam-induced local deformation of the sample is hard to estimate from the low-contrast sample alone, and often requires fiducial gold bead markers. The state-of-the-art approach for deformation estimation uses (semi-)manually labelled marker locations in projection data to fit the parameters of a polynomial deformation model. Manually-labelled marker locations are difficult to obtain when data are noisy or markers overlap in projection data. We propose an alternative mathematical approach…
Aiming at automatic, convenient and non-instrusive motion capture, this paper presents a new generation markerless motion capture technique, the FlyCap system, to capture surface motions of moving characters using multiple autonomous flying cameras (autonomous unmanned aerial vehicles(UAV) each integrated with an RGBD video camera). During data capture, three cooperative flying cameras automatically track and follow the moving target who performs large scale motions in a wide space. We propose a novel non-rigid surface registration method to track and fuse the depth of the three flying cameras…
Michigan State University（密歇根州立大学）；Globus Medical（格罗布斯医疗）
We focus on the problem of timing errors in navigation camera as a case study in a broader problem of the effect of a timing error in cyber-physical systems. These systems rely on the requirement that certain things happen at the same time or certain things happen periodically at some period $T$. However, as these systems get more complex, timing errors can occur between the components thereby violating the assumption about events being simultaneous (or periodic). We consider the problem of a surgical navigation system where optical markers detected in the 2D pictures taken by two cameras are …
Universidade Federal de Minas Gerais (UFMG)（米纳斯吉拉斯联邦大学（UFMG））；Universidade Federal de Alagoas (UFAL)（阿拉戈斯联邦大学（UFAL））
We propose a novel deep metric learning method. Differently from many works on this area, we defined a novel latent space obtained through an autoencoder. The new space, namely S-space, is divided into different regions that describe the positions where pairs of objects are similar/dissimilar. We locate makers to identify these regions. We estimate the similarities between objects through a kernel-based t-student distribution to measure the markers' distance and the new data representation. In our approach, we simultaneously estimate the markers' position in the S-space and represent the objec…
Unmanned aerial vehicles (UAVs) have been used for many applications in recent years, from urban search and rescue, to agricultural surveying, to autonomous underground mine exploration. However, deploying UAVs in tight, indoor spaces, especially close to humans, remains a challenge. One solution, when limited payload is required, is to use micro-UAVs, which pose less risk to humans and typically cost less to replace after a crash. However, micro-UAVs can only carry a limited sensor suite, e.g. a monocular camera instead of a stereo pair or LiDAR, complicating tasks like dense mapping and mark…
