arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1902.09795cs.CV
收藏

基于双流位姿引导的区域包围网的指尖定位

Bi-stream Pose Guided Region Ensemble Network for Fingertip Localization from Stereo Images

Tsinghua University(清华大学);University College London(伦敦大学学院);Beijing Huajie IMI Technology Co., Ltd.(北京华捷艾米科技有限公司)

Guijin Wang, Cairong Zhang, Xinghao Chen, Xiangyang Ji, Jing-Hao Xue, Hang Wang

摘要:

在人机交互中,准确估计手的姿态尤其是指尖的姿态是非常重要的。然而,传统的指尖定位方法主要依赖于深度图像,因此受到噪声和缺失值的影响。代替深度图像,立体图像还可以提供手部的3D信息并促进3D手部姿态估计。然而,在公开可用的基于立体的手部姿势数据集中,数据集大小、全局视点、手部关节和手部形状存在限制。为了缓解这些限制,促进进一步研究从立体图像的手姿态估计,我们提出了一个新的大规模双目手姿态数据集称为THU-Bi-Hand,提供了一个新的视角指尖定位。在THU-Bi-Hand数据集中,有来自10个受试者的447 k对不同手形的立体图像,具有手腕和五个指尖的准确3D位置注释。该数据集对手部运动范围的限制最小,覆盖了大的全局视点空间和手部关节空间。为了更好地呈现THU-Bi-Hand的指尖定位性能,我们提出了一种新的方案,称为双流位姿引导区域包围网(Bi-Pose-REN)。该方法在先前估计的姿态的指导下,在特征图中的关节点周围提取更具代表性的特征区域。根据手部关节的拓扑结构对特征区域进行分层集成,以回归精确的手部姿态。在THU-Bi-Hand上对Bi-Pose-REN和几种现有的方法进行了评估,以便为进一步的研究提供基准。实验结果表明,我们的新方法在THU-Bi-Hand上取得了最好的性能。

英文摘要:

In human-computer interaction, it is important to accurately estimate the hand pose especially fingertips. However, traditional approaches for fingertip localization mainly rely on depth images and thus suffer considerably from the noise and missing values. Instead of depth images, stereo images can also provide 3D information of hands and promote 3D hand pose estimation. There are nevertheless limitations on the dataset size, global viewpoints, hand articulations and hand shapes in the publicly available stereo-based hand pose datasets. To mitigate these limitations and promote further research on hand pose estimation from stereo images, we propose a new large-scale binocular hand pose dataset called THU-Bi-Hand, offering a new perspective for fingertip localization. In the THU-Bi-Hand dataset, there are 447k pairs of stereo images of different hand shapes from 10 subjects with accurate 3D location annotations of the wrist and five fingertips. Captured with minimal restriction on the range of hand motion, the dataset covers large global viewpoint space and hand articulation space. To better present the performance of fingertip localization on THU-Bi-Hand, we propose a novel scheme termed Bi-stream Pose Guided Region Ensemble Network (Bi-Pose-REN). It extracts more representative feature regions around joint points in the feature maps under the guidance of the previously estimated pose. The feature regions are integrated hierarchically according to the topology of hand joints to regress the refined hand pose. Bi-Pose-REN and several existing methods are evaluated on THU-Bi-Hand so that benchmarks are provided for further research. Experimental results show that our new method has achieved the best performance on THU-Bi-Hand.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑