arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.21054cs.CVcs.AIcs.HCcs.LG
收藏

DexAvatar: 基于手部和身体姿态先验的3D手语重建

DexAvatar: 3D Sign Language Reconstruction with Hand and Body Pose Priors

Monash University(墨尔本大学)

Kaustubh Kundu, Hrishav Bakul Barua, Lucy Robertson-Bell, Zhixi Cai, Kalin Stefanov

AI总结:

DexAvatar通过学习的3D手部和身体先验,实现了对野外单目手语视频中细粒度手部动作和身体运动的高精度重建,提升了手语姿态估计的性能。

摘要:

手语生成的趋势集中在数据驱动的生成方法上,这些方法需要大量精确的2D和3D人体姿态数据才能达到可接受的生成质量。然而,目前大多数手语数据集都是基于视频的,仅限于自动重建的2D人体姿态(即关键点),缺乏准确的3D信息。此外,现有最先进的从手语视频中自动估计3D人体姿态的方法容易受到自遮挡、噪声和运动模糊的影响,导致重建质量差。为此,我们引入了DexAvatar,一种新的框架,旨在从野外单目手语视频中重建生物力学准确的细粒度手部动作和身体运动,通过学习的3D手部和身体先验进行指导。DexAvatar在SGNify动作捕捉数据集上表现强劲,这是唯一可用于此任务的基准,其在身体和手部姿态估计上的改进达到了比最先进方法高出35.11%。本工作的官方网站是:https://github.com/kaustesseract/DexAvatar。

英文摘要:

The trend in sign language generation is centered around data-driven generative methods that require vast amounts of precise 2D and 3D human pose data to achieve an acceptable generation quality. However, currently, most sign language datasets are video-based and limited to automatically reconstructed 2D human poses (i.e., keypoints) and lack accurate 3D information. Furthermore, existing state-of-the-art for automatic 3D human pose estimation from sign language videos is prone to self-occlusion, noise, and motion blur effects, resulting in poor reconstruction quality. In response to this, we introduce DexAvatar, a novel framework to reconstruct bio-mechanically accurate fine-grained hand articulations and body movements from in-the-wild monocular sign language videos, guided by learned 3D hand and body priors. DexAvatar achieves strong performance in the SGNify motion capture dataset, the only benchmark available for this task, reaching an improvement of 35.11% in the estimation of body and hand poses compared to the state-of-the-art. The official website of this work is: https://github.com/kaustesseract/DexAvatar.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑