arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2301.06648cs.CVcs.AI
收藏

动态环境中的神经形态高频3D舞蹈姿态估计

Neuromorphic High-Frequency 3D Dancing Pose Estimation in Dynamic Environment

University of California San Diego(加州大学圣地亚哥分校);University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校)

Zhongyang Zhang, Kaidong Chai, Haowen Yu, Ramzi Majaj, Francesca Walsh, Edward Wang, Upal Mahbub, Hava Siegelmann, Donghyun Kim, Tauhidur Rahman

AI总结:

本文提出基于事件相机的三维高频人体姿态估计系统YeLan,通过构建首个事件相机舞蹈数据集和物理感知模拟器,解决了低光照和动态背景下舞蹈姿态估计的鲁棒性问题。

摘要:

作为一项全球广受喜爱的运动,舞蹈如今正被整合到传统和基于虚拟现实的游戏平台中,这为技术介导的舞蹈领域开辟了新的机遇。这些平台主要依赖被动且连续的人体姿态估计作为输入捕捉机制。现有解决方案主要基于RGB或RGB-深度相机用于舞蹈游戏。前者在低光照条件下因运动模糊和低灵敏度而表现不佳,而后者功耗过高、帧率低且工作距离有限。凭借超低延迟、高能效和宽动态范围特性,事件相机是克服这些缺点的有前景的解决方案。我们提出YeLan,一个基于事件相机的三维高频人体姿态估计(HPE)系统,能够在低光照条件和动态背景下正常工作。我们收集了世界上第一个事件相机舞蹈数据集,并开发了一个完全可定制的运动到事件物理感知模拟器。YeLan在这些具有挑战性的条件下优于基线模型,并展示了对不同类型服装、背景运动、视角、遮挡和光照波动的鲁棒性。

英文摘要:

As a beloved sport worldwide, dancing is getting integrated into traditional and virtual reality-based gaming platforms nowadays. It opens up new opportunities in the technology-mediated dancing space. These platforms primarily rely on passive and continuous human pose estimation as an input capture mechanism. Existing solutions are mainly based on RGB or RGB-Depth cameras for dance games. The former suffers in low-lighting conditions due to the motion blur and low sensitivity, while the latter is too power-hungry, has a low frame rate, and has limited working distance. With ultra-low latency, energy efficiency, and wide dynamic range characteristics, the event camera is a promising solution to overcome these shortcomings. We propose YeLan, an event camera-based 3-dimensional high-frequency human pose estimation(HPE) system that survives low-lighting conditions and dynamic backgrounds. We collected the world's first event camera dance dataset and developed a fully customizable motion-to-event physics-aware simulator. YeLan outperforms the baseline models in these challenging conditions and demonstrated robustness against different types of clothing, background motion, viewing angle, occlusion, and lighting fluctuations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑