arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1911.09231cs.RO
收藏

基于单幅图像的摄像机-机器人位姿估计

Camera-to-Robot Pose Estimation from a Single Image

NVIDIA(英伟达);Carnegie Mellon University(卡内基梅隆大学)

Timothy E. Lee, Jonathan Tremblay, Thang To, Jia Cheng, Terry Mosier, Oliver Kroemer, Dieter Fox, Stan Birchfield

摘要:

我们提出了一种方法,用于估计相对于机器人使用单个RGB图像的机器人的外部相机的姿态。图像由深度神经网络处理,以检测与机器人相关联的关键点(如关节)的2D投影。该网络完全基于模拟数据进行训练,使用域随机化来弥合现实差距。视角n-点(PPENS),然后用于恢复相机extrinsics,假设相机的内在和关节配置的机器人操作器是已知的。与经典的手眼校准系统不同,我们的方法不需要离线校准步骤。相反,它是能够计算的相机extrinsics从一个单一的帧,从而打开在线校准的可能性。我们展示了三种不同的机器人和相机传感器的实验结果,证明我们的方法能够用单个帧实现精度,这与使用多个帧的经典离线手眼校准相当。使用来自静态姿势的附加帧,精度进一步提高。代码,数据集和预训练模型的三个广泛使用的机器人操纵器。

英文摘要:

We present an approach for estimating the pose of an external camera with respect to a robot using a single RGB image of the robot. The image is processed by a deep neural network to detect 2D projections of keypoints (such as joints) associated with the robot. The network is trained entirely on simulated data using domain randomization to bridge the reality gap. Perspective-n-point (PnP) is then used to recover the camera extrinsics, assuming that the camera intrinsics and joint configuration of the robot manipulator are known. Unlike classic hand-eye calibration systems, our method does not require an off-line calibration step. Rather, it is capable of computing the camera extrinsics from a single frame, thus opening the possibility of on-line calibration. We show experimental results for three different robots and camera sensors, demonstrating that our approach is able to achieve accuracy with a single frame that is comparable to that of classic off-line hand-eye calibration using multiple frames. With additional frames from a static pose, accuracy improves even further. Code, datasets, and pretrained models for three widely-used robot manipulators are made available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑