arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.04066cs.CV
收藏

TEACH:三维人体的时间动作合成

TEACH: Temporal Action Composition for 3D Humans

Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);École des Ponts(巴黎桥路学院);Univ Gustave Eiffel(古斯塔夫·埃菲尔大学);CNRS(法国国家科学研究中心)

Nikos Athanasiou, Mathis Petrovich, Michael J. Black, Gül Varol

摘要:

给定一系列自然语言描述,我们的任务是生成与文本语义相对应的三维人体运动,并遵循指令的时间顺序。特别是,我们的目标是使一系列的行动,我们称之为时间的行动组成的合成。当前的文本条件运动合成技术仅将单个动作或单个句子作为输入。这部分是由于缺乏包含动作序列的合适的训练数据,但也由于其非自回归模型公式化的计算复杂性,其不能很好地扩展到长序列。在这项工作中,我们解决这两个问题。首先,我们利用最近的BABEL运动文本集合,它有广泛的标记动作,其中许多发生在它们之间的过渡序列。接下来,我们设计了一个基于transformer的方法,它在一个动作中非自回归地操作,但在动作序列中自回归地操作。在我们的实验中,与多个基线相比,这种分层公式证明是有效的。我们的方法,被称为“TEACH为人类运动的临时动作组成”,产生逼真的人类运动的各种各样的行动和时间组成的语言描述。为了鼓励在这个新任务上的工作,我们在我们的$\href{teach.is.tue.mpg.de}{\text{website}}$上提供我们的代码用于研究目的。

英文摘要:

Given a series of natural language descriptions, our task is to generate 3D human motions that correspond semantically to the text, and follow the temporal order of the instructions. In particular, our goal is to enable the synthesis of a series of actions, which we refer to as temporal action composition. The current state of the art in text-conditioned motion synthesis only takes a single action or a single sentence as input. This is partially due to lack of suitable training data containing action sequences, but also due to the computational complexity of their non-autoregressive model formulation, which does not scale well to long sequences. In this work, we address both issues. First, we exploit the recent BABEL motion-text collection, which has a wide range of labeled actions, many of which occur in a sequence with transitions between them. Next, we design a Transformer-based approach that operates non-autoregressively within an action, but autoregressively within the sequence of actions. This hierarchical formulation proves effective in our experiments when compared with multiple baselines. Our approach, called TEACH for "TEmporal Action Compositions for Human motions", produces realistic human motions for a wide variety of actions and temporal compositions from language descriptions. To encourage work on this new task, we make our code available for research purposes at our $\href{teach.is.tue.mpg.de}{\text{website}}$.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑