arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.07044cs.CV
收藏

基于RGB视频的无约束关节手目标重建

Towards unconstrained joint hand-object reconstruction from RGB videos

Inria(法国国家信息与自动化研究所);CNRS(法国国家科学研究中心);PSL Research University(巴黎文理研究大学);Univ Gustave Eiffel(古斯塔夫·埃菲尔大学)

Yana Hasson, Gül Varol, Ivan Laptev, Cordelia Schmid

摘要:

我们的工作旨在从单目视频中获得手部和操纵对象的3D重建。重建手物体的操作有很大的潜力,机器人和人类的示范学习。然而,解决这个问题的监督学习方法需要3D监督,并且仍然限于受限的实验室设置和3D地面实况可用的模拟器。在本文中,我们首先提出了一个学习自由拟合方法的手对象重建,可以无缝地处理双手对象的相互作用。我们的方法依赖于与目标检测,手部姿势估计和实例分割的常见方法获得的线索。我们定量地评估我们的方法,并表明它可以应用于不同难度的数据集,其中训练数据是不可用的。

英文摘要:

Our work aims to obtain 3D reconstruction of hands and manipulated objects from monocular videos. Reconstructing hand-object manipulations holds a great potential for robotics and learning from human demonstrations. The supervised learning approach to this problem, however, requires 3D supervision and remains limited to constrained laboratory settings and simulators for which 3D ground truth is available. In this paper we first propose a learning-free fitting approach for hand-object reconstruction which can seamlessly handle two-hand object interactions. Our method relies on cues obtained with common methods for object detection, hand pose estimation and instance segmentation. We quantitatively evaluate our approach and show that it can be applied to datasets with varying levels of difficulty for which training data is unavailable.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑