arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.05168cs.CV
收藏

SeqHAND:基于RGB序列的3D手部姿态和形状估计

SeqHAND:RGB-Sequence-Based 3D Hand Pose and Shape Estimation

Seoul National University(首尔大学);University of Birmingham(伯明翰大学)

John Yang, Hyung Jin Chang, Seungeui Lee, Nojun Kwak

摘要:

基于RGB图像的三维手势估计已经研究了很长时间。然而,大多数研究都是基于独立的静态图像进行逐帧估计。在本文中,我们试图不仅考虑手的外观,而且将运动中的手的时间运动信息纳入学习框架中,以获得更好的3D手部姿势估计性能,这导致需要具有连续RGB手部图像的大规模数据集。我们提出了一种新的方法,生成一个合成的数据集,模仿自然的人类手部动作,通过重新设计的注释现存的静态手部姿势数据集的姿势流。与生成的数据集,我们训练一个新提出的经常性框架,利用视觉时间的功能,从序列图像的合成手在运动中,并强调时间平滑的时间一致性约束的估计。我们的新的训练策略分离的框架的递归层在域微调从合成到真实允许保存的视觉-时间特征从顺序合成的手图像。因此,顺序估计的手部姿势产生自然和平滑的手部运动,这导致更鲁棒的估计。我们表明,利用时间信息的3D手姿态估计显着提高了一般的姿态估计优于国家的最先进的方法在手姿态估计基准的实验。

英文摘要:

3D hand pose estimation based on RGB images has been studied for a long time. Most of the studies, however, have performed frame-by-frame estimation based on independent static images. In this paper, we attempt to not only consider the appearance of a hand but incorporate the temporal movement information of a hand in motion into the learning framework for better 3D hand pose estimation performance, which leads to the necessity of a large scale dataset with sequential RGB hand images. We propose a novel method that generates a synthetic dataset that mimics natural human hand movements by re-engineering annotations of an extant static hand pose dataset into pose-flows. With the generated dataset, we train a newly proposed recurrent framework, exploiting visuo-temporal features from sequential images of synthetic hands in motion and emphasizing temporal smoothness of estimations with a temporal consistency constraint. Our novel training strategy of detaching the recurrent layer of the framework during domain finetuning from synthetic to real allows preservation of the visuo-temporal features learned from sequential synthetic hand images. Hand poses that are sequentially estimated consequently produce natural and smooth hand movements which lead to more robust estimations. We show that utilizing temporal information for 3D hand pose estimation significantly enhances general pose estimations by outperforming state-of-the-art methods in experiments on hand pose estimation benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑