视角手部交互物体姿态估计的基准与挑战
Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects
AI总结：
本文提出HANDS23挑战，分析了以自身为中心视角下手-物体三维重建的挑战与方法，揭示了快速手部运动和狭窄视角等难以解决的问题。
摘要：
我们通过双手与世界互动，并通过自身（以自身为中心）的视角来观察它。从以自身为中心的视角对这种交互进行整体三维理解对于机器人、AR/VR、动作识别和运动生成等任务至关重要。准确地在三维空间中重建这种交互具有挑战性，因为存在严重的遮挡、视角偏差、相机畸变以及由于头部运动引起的运动模糊。为此，我们基于AssemblyHands和ARCTIC数据集设计了HANDS23挑战，并精心设计了训练和测试分割。基于排行榜上最佳提交方法和最近基线的结果，我们对三维手（-物体）重建任务进行了深入分析。我们的分析证明了针对以自身为中心相机特有的畸变进行处理、采用高容量的Transformer来学习复杂的手-物体交互、以及融合不同视角的预测是有效的。我们的研究进一步揭示了无法用最先进方法解决的具有挑战性的场景，例如快速的手部运动、从狭窄的以自身为中心视角重建物体、以及两只手与物体之间的紧密接触。我们的努力将丰富社区的知识基础，并促进未来关于以自身为中心的手-物体交互的手研究。
英文摘要：
We interact with the world with our hands and see it through our own (egocentric) perspective. A holistic 3Dunderstanding of such interactions from egocentric views is important for tasks in robotics, AR/VR, action recognition and motion generation. Accurately reconstructing such interactions in 3D is challenging due to heavy occlusion, viewpoint bias, camera distortion, and motion blur from the head movement. To this end, we designed the HANDS23 challenge based on the AssemblyHands and ARCTIC datasets with carefully designed training and testing splits. Based on the results of the top submitted methods and more recent baselines on the leaderboards, we perform a thorough analysis on 3D hand(-object) reconstruction tasks. Our analysis demonstrates the effectiveness of addressing distortion specific to egocentric cameras, adopting high-capacity transformers to learn complex hand-object interactions, and fusing predictions from different views. Our study further reveals challenging scenarios intractable with state-of-the-art methods, such as fast hand motion, object reconstruction from narrow egocentric views, and close contact between two hands and objects. Our efforts will enrich the community's knowledge foundation and facilitate future hand studies on egocentric hand-object interactions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
