arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.14531cs.CV
收藏

Uni6D:一个统一的无投影分解的CNN框架

Uni6D: A Unified CNN Framework without Projection Breakdown for 6D Pose Estimation

SenseTime Research(商汤科技研究院);University of Chinese Academy of Sciences(中国科学院大学)

Xiaoke Jiang, Donghai Li, Hao Chen, Ye Zheng, Rui Zhao, Liwei Wu

摘要:

随着RGB-D传感器变得越来越便宜,使用RGB-D图像来获得高精度的6D姿态估计结果成为更好的选择。最先进的方法通常使用不同的主干来提取RGB和深度图像的特征。他们使用2D CNN用于RGB图像,使用每像素点云网络用于深度数据,以及使用融合网络进行特征融合。我们发现,使用两个独立的骨干网的根本原因是“投影故障”的问题。在深度图像平面中,物理世界的投影3D结构由1D深度值及其内置的2D像素坐标(UV)保留。修改UV的任何空间变换(例如CNN管道中的调整大小、翻转、裁剪或池化操作)都会断开像素值和UV坐标之间的绑定。因此,3D结构不再被修改的深度图像或特征保留。为了解决这个问题,我们提出了一种简单而有效的方法,称为Uni 6D,它明确地将额外的UV数据与RGB-D图像一起作为输入。我们的方法具有统一的CNN框架,用于具有单个CNN骨干的6D姿态估计。特别是,我们的方法的架构是基于Mask R-CNN,带有两个额外的头,一个名为RT头,用于直接预测6D姿态,另一个名为abc头,用于指导网络将可见点映射到3D模型中的坐标作为辅助模块。这种端到端方法平衡了简单性和准确性,实现了与最先进技术相当的准确性,并在YCB-Video数据集上实现了7.2倍的推理速度。

英文摘要:

As RGB-D sensors become more affordable, using RGB-D images to obtain high-accuracy 6D pose estimation results becomes a better option. State-of-the-art approaches typically use different backbones to extract features for RGB and depth images. They use a 2D CNN for RGB images and a per-pixel point cloud network for depth data, as well as a fusion network for feature fusion. We find that the essential reason for using two independent backbones is the "projection breakdown" problem. In the depth image plane, the projected 3D structure of the physical world is preserved by the 1D depth value and its built-in 2D pixel coordinate (UV). Any spatial transformation that modifies UV, such as resize, flip, crop, or pooling operations in the CNN pipeline, breaks the binding between the pixel value and UV coordinate. As a consequence, the 3D structure is no longer preserved by a modified depth image or feature. To address this issue, we propose a simple yet effective method denoted as Uni6D that explicitly takes the extra UV data along with RGB-D images as input. Our method has a Unified CNN framework for 6D pose estimation with a single CNN backbone. In particular, the architecture of our method is based on Mask R-CNN with two extra heads, one named RT head for directly predicting 6D pose and the other named abc head for guiding the network to map the visible points to their coordinates in the 3D model as an auxiliary module. This end-to-end approach balances simplicity and accuracy, achieving comparable accuracy with state of the arts and 7.2x faster inference speed on the YCB-Video dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑