arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2306.06367cs.CVcs.AI
收藏

用于运动插值的打乱自回归(Shuffled Autoregression)方法

Shuffled Autoregression For Motion Interpolation

Tsinghua University(清华大学);Zhejiang University(浙江大学);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心)

Shuo Huang, Jia Jia, Zongxin Yang, Wei Wang, Haozhe Wu, Yi Yang, Junliang Xing

AI总结:

针对输入为孤立姿态的运动插值任务,提出打乱自回归框架与时空运动Transformer,利用帧间灵活依赖提升性能,效果优于现有方法且可扩展至多场景。

摘要:

本研究旨在为运动插值任务提供一种深度学习解决方案。以往研究通过几何权重函数求解该问题,另有部分工作针对以连续姿态序列为输入的不同问题设置提出了神经网络方法。但运动插值是更为复杂的问题,其输入为孤立姿态(例如仅一个起始姿态和一个结束姿态)。这些深度学习方法应用于运动插值时性能有限,因为它们无法像原始几何公式那样利用插值帧之间的灵活依赖关系。为实现这一插值特性,我们提出了名为Shuffled AutoRegression(打乱自回归)的新型框架,该框架将自回归扩展为可按任意(打乱)顺序生成,并将任意帧间依赖关系建模为有向无环图。我们进一步提出了一种构建特定类型依赖图的方法,将三个阶段整合为端到端的时空运动Transformer。在当前最大的数据集之一上的实验结果表明,我们的模型仅需一个起始帧和一个结束帧即可生成生动连贯的运动,且性能大幅优于对比方法。所提模型还可扩展至多关键帧运动插值任务及其他领域的插值问题。

英文摘要:

This work aims to provide a deep-learning solution for the motion interpolation task. Previous studies solve it with geometric weight functions. Some other works propose neural networks for different problem settings with consecutive pose sequences as input. However, motion interpolation is a more complex problem that takes isolated poses (e.g., only one start pose and one end pose) as input. When applied to motion interpolation, these deep learning methods have limited performance since they do not leverage the flexible dependencies between interpolation frames as the original geometric formulas do. To realize this interpolation characteristic, we propose a novel framework, referred to as \emph{Shuffled AutoRegression}, which expands the autoregression to generate in arbitrary (shuffled) order and models any inter-frame dependencies as a directed acyclic graph. We further propose an approach to constructing a particular kind of dependency graph, with three stages assembled into an end-to-end spatial-temporal motion Transformer. Experimental results on one of the current largest datasets show that our model generates vivid and coherent motions from only one start frame to one end frame and outperforms competing methods by a large margin. The proposed model is also extensible to multiple keyframes' motion interpolation tasks and other areas' interpolation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑