结合循环神经网络和对抗训练的人体运动合成与控制
Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control
摘要:
本文介绍了一种新的用于人体运动合成和控制的生成式深度学习网络。我们的核心思想是将循环神经网络(RNN)和对抗训练结合起来,用于人体运动建模。我们首先描述了一种从预先记录的运动数据训练RNN模型的有效方法。我们用长短期记忆(LSTM)细胞实现循环神经网络,因为它们能够处理人类运动中存在的非线性动力学和长期时间依赖性。接下来,我们使用对抗损失训练一个细化网络,类似于生成对抗网络(GANs),使得细化的运动序列与使用判别网络的真实运动捕捉数据无法区分。我们将联系人信息嵌入到生成式深度学习模型中,以进一步提高生成式模型的性能。由此产生的模型是有吸引力的运动合成和控制,因为它是紧凑的,接触感知,并可以生成无限数量的自然看起来无限长的运动。我们的实验表明,由我们的深度学习模型生成的运动总是高度逼真的,可以与高质量的运动捕捉数据相媲美。我们通过探索各种应用程序,从随机运动合成,在线/离线运动控制和运动过滤,展示了我们的模型的功能和有效性。通过与基线模型的比较,我们展示了生成模型的优越性。
英文摘要:
This paper introduces a new generative deep learning network for human motion synthesis and control. Our key idea is to combine recurrent neural networks (RNNs) and adversarial training for human motion modeling. We first describe an efficient method for training a RNNs model from prerecorded motion data. We implement recurrent neural networks with long short-term memory (LSTM) cells because they are capable of handling nonlinear dynamics and long term temporal dependencies present in human motions. Next, we train a refiner network using an adversarial loss, similar to Generative Adversarial Networks (GANs), such that the refined motion sequences are indistinguishable from real motion capture data using a discriminative network. We embed contact information into the generative deep learning model to further improve the performance of our generative model. The resulting model is appealing to motion synthesis and control because it is compact, contact-aware, and can generate an infinite number of naturally looking motions with infinite lengths. Our experiments show that motions generated by our deep learning model are always highly realistic and comparable to high-quality motion capture data. We demonstrate the power and effectiveness of our models by exploring a variety of applications, ranging from random motion synthesis, online/offline motion control, and motion filtering. We show the superiority of our generative model by comparison against baseline models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。