arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2008.08171cs.CVcs.GR
收藏

学习使用Transformer生成各种舞蹈动作

Learning to Generate Diverse Dance Motions with Transformer

University of Southern California(南加州大学);USC Institute for Creative Technologies(南加州大学创新技术研究所);Beihang University(北京航空航天大学);University of Toronto(多伦多大学);Vector Institute(向量研究所);Pinscreen

Jiaman Li, Yihang Yin, Hang Chu, Yi Zhou, Tingwu Wang, Sanja Fidler, Hao Li

摘要:

随着疫情的持续,虚拟音乐会和使用音乐家数字化表演的现场活动在大型多人在线世界中越来越受欢迎。然而,精心设计的舞蹈动作是非常复杂的动画,将涉及昂贵和繁琐的生产过程。除了使用复杂的动作捕捉系统外,它通常还需要动画师、舞蹈演员和编舞者之间的协作。我们介绍了一个完整的舞蹈动作合成系统,它可以生成复杂的和高度多样化的舞蹈序列给定的输入音乐序列。由于动作捕捉数据是有限的舞蹈动作和风格的范围内,我们介绍了一个庞大的舞蹈动作数据集,是从YouTube视频创建的。我们还提出了一种新的双流运动Transformer生成模型,它可以生成运动序列具有很高的灵活性。我们还介绍了新的评价指标的合成舞蹈动作的质量,并证明我们的系统可以优于国家的最先进的方法。我们的系统为虚拟音乐会提供适合大量人群的高质量动画,也可以作为专业动画管道的参考。最重要的是,我们证明了大量的在线视频可以有效地训练舞蹈动作模型。

英文摘要:

With the ongoing pandemic, virtual concerts and live events using digitized performances of musicians are getting traction on massive multiplayer online worlds. However, well choreographed dance movements are extremely complex to animate and would involve an expensive and tedious production process. In addition to the use of complex motion capture systems, it typically requires a collaborative effort between animators, dancers, and choreographers. We introduce a complete system for dance motion synthesis, which can generate complex and highly diverse dance sequences given an input music sequence. As motion capture data is limited for the range of dance motions and styles, we introduce a massive dance motion data set that is created from YouTube videos. We also present a novel two-stream motion transformer generative model, which can generate motion sequences with high flexibility. We also introduce new evaluation metrics for the quality of synthesized dance motions, and demonstrate that our system can outperform state-of-the-art methods. Our system provides high-quality animations suitable for large crowds for virtual concerts and can also be used as reference for professional animation pipelines. Most importantly, we show that vast online videos can be effective in training dance motion models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑