基于长视野建模的人体运动时空流形学习
Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling
摘要：
数据驱动的人体运动建模在计算机图形学和计算机视觉应用中是普遍存在的，例如合成逼真的运动或识别动作。最近的研究表明，这些问题可以通过使用深度学习学习自然运动流形来解决，以解决传统数据驱动方法的缺点。然而，由于两个原因，先前的方法可能是次优的。首先，骨架信息没有被充分利用于特征提取。与图像不同，很难以深度网络可以应用的方式定义骨骼运动中的空间邻近度。其次，运动是具有强多模态时间相关性的时间序列数据。一个帧之后可能有几个候选帧，导致不同的运动;长距离依赖性存在于开始时的多个帧与之后的多个帧相关的地方。无效的建模会低估多模态和方差，导致无特征的平均运动，或者高估它们，导致抖动运动。在本文中，我们提出了一种新的深度网络，通过创建一个适用于许多应用的自然运动流形来应对这些挑战。该网络具有用于特征提取的新的空间分量。它还配备了一个新的批量预测模型，可以一次预测大量的帧，这样就可以采用长期的基于时间的目标函数来正确地学习运动多模态和方差。使用我们的系统，可以使用开环设置来预测/合成长时间运动，其中运动准确地保持动态。它还可以用于对损坏的运动进行去噪，并利用给定的控制信号合成新的运动。我们证明，我们的系统可以创建优越的结果相比，现有的工作在多个应用程序。
英文摘要：
Data-driven modeling of human motions is ubiquitous in computer graphics and computer vision applications, such as synthesizing realistic motions or recognizing actions. Recent research has shown that such problems can be approached by learning a natural motion manifold using deep learning to address the shortcomings of traditional data-driven approaches. However, previous methods can be sub-optimal for two reasons. First, the skeletal information has not been fully utilized for feature extraction. Unlike images, it is difficult to define spatial proximity in skeletal motions in the way that deep networks can be applied. Second, motion is time-series data with strong multi-modal temporal correlations. A frame could be followed by several candidate frames leading to different motions; long-range dependencies exist where a number of frames in the beginning correlate to a number of frames later. Ineffective modeling would either under-estimate the multi-modality and variance, resulting in featureless mean motion or over-estimate them resulting in jittery motions. In this paper, we propose a new deep network to tackle these challenges by creating a natural motion manifold that is versatile for many applications. The network has a new spatial component for feature extraction. It is also equipped with a new batch prediction model that predicts a large number of frames at once, such that long-term temporally-based objective functions can be employed to correctly learn the motion multi-modality and variances. With our system, long-duration motions can be predicted/synthesized using an open-loop setup where the motion retains the dynamics accurately. It can also be used for denoising corrupted motions and synthesizing new motions with given control signals. We demonstrate that our system can create superior results comparing to existing work in multiple applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
