arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.06114cs.CVcs.GR
收藏

迈向非结构化无标记光学动捕:视频大有帮助!

Towards Unstructured Unlabeled Optical Mocap: A Video Helps!

Texas A&M University(德克萨斯农工大学);University of Macau(澳门大学)

Nicholas Milef, John Keyser, Shu Kong

AI总结:

针对标记点可任意放置且无标注的UUO光学动捕,本文利用单目RGB视频提供人体先验,提出联合标记点识别标注、姿态估计与身体重建的优化流程,并在三个基准上显著超越现有方法。

摘要:

光学动作捕捉(mocap)需要从反光标记点精确重建人体,包括姿态和形状。在典型的动捕设置中,标记点标注是一项重要但繁琐且容易出错的步骤。以往工作表明,可以通过使用定义特定标记点放置位置的结构化模板来自动化标记点标注,但这会带来额外的录制约束。我们建议放宽这些约束,并解决非结构化无标记光学(UUO)动捕问题。与典型的动捕设置(要么标注标记点,要么按照结构化布局放置标记点)相比,UUO动捕中的标记点可以放置在身体的任何位置,甚至可以只放在某一特定肢体上(例如用于生物力学研究的右腿),因此具有更重要的实际意义。它也更具挑战性。为了解决UUO动捕问题,我们利用单个RGB相机拍摄的单目视频,无需相机标定。在该视频上,我们运行一种现成方法来重建和跟踪人体,从而提供有关人体姿态和形状的强视觉先验。结合视频和UUO标记点,我们提出了一条面向标记点识别、标记点标注、人体姿态估计和人体重建的优化流程。我们的技术创新包括用于优化全局朝向的多假设测试,以及用于更好地优化身体表面的标记点定位和标记点-部位匹配。我们进行了大量实验,将本方法与最先进方法进行定量比较,包括仅使用标记点的动捕和仅使用视频的人体/形状重建。实验表明,在三个公认基准数据集上,本方法在全身和部分身体重建方面均显著优于现有方法。

英文摘要:

Optical motion capture (mocap) requires accurately reconstructing the human body from retroreflective markers, including pose and shape. In a typical mocap setting, marker labeling is an important but tedious and error-prone step. Previous work has shown that marker labeling can be automated by using a structured template defining specific marker placements, but this places additional recording constraints. We propose to relax these constraints and solve for Unstructured Unlabeled Optical (UUO) mocap. Compared to the typical mocap setting that either labels markers or places them w.r.t a structured layout, markers in UUO mocap can be placed anywhere on the body and even on one specific limb (e.g., right leg for biomechanics research), hence it is of more practical significance. It is also more challenging. To solve UUO mocap, we exploit a monocular video captured by a single RGB camera, which does not require camera calibration. On this video, we run an off-the-shelf method to reconstruct and track a human individual, giving strong visual priors of human body pose and shape. With both the video and UUO markers, we propose an optimization pipeline towards marker identification, marker labeling, human pose estimation, and human body reconstruction. Our technical novelties include multiple hypothesis testing to optimize global orientation, and marker localization and marker-part matching to better optimize for body surface. We conduct extensive experiments to quantitatively compare our method against state-of-the-art approaches, including marker-only mocap and video-only human body/shape reconstruction. Experiments demonstrate that our method resoundingly outperforms existing methods on three established benchmark datasets for both full-body and partial-body reconstruction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑