手部可见性检测器：针对手部关键点的逐点可见性估计

中文摘要 AI 辅助

手部姿态估计（Hand Pose Estimation, HPE）是增强现实/虚拟现实（AR/VR）、机器人等多种应用的基础技术。在这些应用中，图像中每个手关节的可见性对于评估遮挡情况下估计结果的可靠性至关重要。然而，大多数现有的HPE方法仅输出关节位置，未明确标注其可见性。尽管部分方法会考虑遮挡或可见性，但可见性估计主要被用作提升姿态估计的辅助信号。据我们所知，将逐关节手部可见性估计作为独立任务进行系统研究的工作尚未出现。本研究提出了Hand Visibility Detector（手部可见性检测器），用于估计单个手关节的可见性，并首次将可见性估计作为独立任务开展系统研究。我们发现，利用在大规模数据上预训练的HPE模型的先验知识作为骨干网络，可在该任务中实现高性能。我们进一步通过2D关键点的多视图三角化，在3D手部姿态标注的下游任务中验证了Hand Visibility Detector的实用性，结果表明，可见性加权三角化可降低重投影误差。本方法已作为可直接使用的软件包发布，代码和演示可在指定URL获取。

英文摘要

Hand Pose Estimation (HPE) is a fundamental technology for various applications such as AR/VR and robotics. In these applications, the visibility of each hand joint in the image is crucial for assessing the reliability of estimation results under occlusion. However, most existing HPE methods output joint positions without explicitly indicating their visibility. Although some methods account for occlusion or visibility, visibility estimation has mainly been used as an auxiliary signal for improving pose estimation. To our knowledge, per-joint hand visibility estimation has not been systematically studied as a standalone task. In this work, we propose Hand Visibility Detector, a model for estimating the visibility of individual hand joints, and present the first systematic investigation of visibility estimation as an independent task. We show that leveraging the prior knowledge of HPE models pretrained on large-scale data as a backbone yields high performance in this task. We further demonstrate the utility of Hand Visibility Detector on a downstream task of 3D hand pose annotation via multi-view triangulation of 2D keypoints, showing that visibility-weighted triangulation reduces reprojection error. Our method is released as a ready-to-use package, and the code and demo are available at https://github.com/ryhara/hand_visibility_detector .

发表机构

• OMRON SINIC X Corporation（欧姆龙SINIC X公司）
• The University of Tokyo（东京大学）
• Keio University（庆应义塾大学）
• National Institute of Advanced Industrial Science Technology (AIST)（日本产业技术综合研究所）

机构由 AI 辅助整理，请以论文原文为准。

补充信息

备注

Code: https://github.com/ryhara/hand_visibility_detector