arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.11703cs.CV
收藏

基于模型的自监督学习手部三维重建

Model-based 3D Hand Reconstruction via Self-Supervised Learning

Wuhan University(武汉大学);Tencent AI Lab(腾讯人工智能实验室);Tencent(腾讯);State University of New York at Buffalo(纽约州立大学布法罗分校)

Yujin Chen, Zhigang Tu, Di Kang, Linchao Bao, Ying Zhang, Xuefei Zhe, Ruizhi Chen, Junsong Yuan

摘要:

从单视图RGB图像重建3D手是具有挑战性的,由于各种手配置和深度模糊。为了从单目图像可靠地重建3D手,大多数最先进的方法在训练阶段严重依赖于3D注释,但是获得3D注释是昂贵的。为了减轻对标记的训练数据的依赖,我们提出了S2HAND,一个自我监督的3D手部重建网络,可以联合估计姿势,形状,纹理和相机的观点。具体来说,我们通过易于访问的2D检测到的关键点从输入图像中获得几何线索。为了从这些嘈杂的几何线索中学习准确的手部重建模型,我们利用2D和3D表示之间的一致性,并提出了一组新的损失来合理化神经网络的输出。我们首次证明了在不依赖手动注释的情况下训练精确的3D手部重建网络的可行性。我们的实验表明,所提出的方法实现了与最近的全监督方法相当的性能,同时使用更少的监督数据。

英文摘要:

Reconstructing a 3D hand from a single-view RGB image is challenging due to various hand configurations and depth ambiguity. To reliably reconstruct a 3D hand from a monocular image, most state-of-the-art methods heavily rely on 3D annotations at the training stage, but obtaining 3D annotations is expensive. To alleviate reliance on labeled training data, we propose S2HAND, a self-supervised 3D hand reconstruction network that can jointly estimate pose, shape, texture, and the camera viewpoint. Specifically, we obtain geometric cues from the input image through easily accessible 2D detected keypoints. To learn an accurate hand reconstruction model from these noisy geometric cues, we utilize the consistency between 2D and 3D representations and propose a set of novel losses to rationalize outputs of the neural network. For the first time, we demonstrate the feasibility of training an accurate 3D hand reconstruction network without relying on manual annotations. Our experiments show that the proposed method achieves comparable performance with recent fully-supervised methods while using fewer supervision data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑