日常自我中心图像中的3D手部姿态估计
3D Hand Pose Estimation in Everyday Egocentric Images
AI总结：
针对日常自我中心图像3D手部姿态估计中透视畸变和缺乏3D标注的挑战，系统研究了不同实践的影响，提出WildHands系统，在多个数据集上实现优异的零样本性能，且模型更小、训练数据更少。
摘要：
由于几个原因，日常自我中心图像中的3D手部姿态估计具有挑战性：视觉信号差（交互物体的遮挡、低分辨率和运动模糊）、大透视畸变（手靠近相机）以及受控环境之外缺乏3D标注。虽然现有方法通常使用手部裁剪作为输入来关注细粒度视觉信息以应对差的视觉信号，但野外环境中透视畸变和缺乏3D标注带来的挑战尚未得到系统研究。我们关注这一空白，并探索不同实践的影响，即裁剪作为输入、结合相机信息、辅助监督、扩大数据集。我们提供了几个适用于卷积和transformer模型的见解，从而带来更好的性能。基于我们的发现，我们还提出了WildHands，一个用于日常自我中心图像中3D手部姿态估计的系统。在4个多样化数据集（H2O、AssemblyHands、Epic-Kitchens、Ego-Exo4D）上的零样本评估证明了我们方法在2D和3D指标上的有效性，我们比过去方法高出7.4% - 66%。在系统级比较中，WildHands在ARCTIC自我中心分割上实现了最佳3D手部姿态，在所有指标上优于FrankMocap，并在6个指标中的3个上优于HaMeR，同时模型小10倍且训练数据少5倍。
英文摘要：
3D hand pose estimation in everyday egocentric images is challenging for several reasons: poor visual signal (occlusion from the object of interaction, low resolution & motion blur), large perspective distortion (hands are close to the camera), and lack of 3D annotations outside of controlled settings. While existing methods often use hand crops as input to focus on fine-grained visual information to deal with poor visual signal, the challenges arising from perspective distortion and lack of 3D annotations in the wild have not been systematically studied. We focus on this gap and explore the impact of different practices, i.e. crops as input, incorporating camera information, auxiliary supervision, scaling up datasets. We provide several insights that are applicable to both convolutional and transformer models leading to better performance. Based on our findings, we also present WildHands, a system for 3D hand pose estimation in everyday egocentric images. Zero-shot evaluation on 4 diverse datasets (H2O, AssemblyHands, Epic-Kitchens, Ego-Exo4D) demonstrate the effectiveness of our approach across 2D and 3D metrics, where we beat past methods by 7.4% - 66%. In system level comparisons, WildHands achieves the best 3D hand pose on ARCTIC egocentric split, outperforms FrankMocap across all metrics and HaMeR on 3 out of 6 metrics while being 10x smaller and trained on 5x less data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
