arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.16016cs.CV
收藏

UDE:一个用于人体运动生成的统一驱动引擎

UDE: A Unified Driving Engine for Human Motion Generation

Zixiang Zhou, Baoyuan Wang

摘要:

生成可控制和可编辑的人体运动序列是3D虚拟形象生成中的一个关键挑战。长期以来,生成和动画人体运动一直是一项劳动密集型的工作,直到最近基于学习的方法被开发和应用。然而,这些方法仍然是任务特定的或模态特定的\cite {ahuja 2019 language 2 pose} \cite{ghosh2021synthesis} \cite{ferreira2021learning} \cite{li2021ai}。在本文中,我们提出了“UDE”,第一个统一的驱动引擎,使生成人类运动序列从自然语言或音频序列(见图~)。\ref{fig:teaser})。具体而言,UDE由以下关键组件组成:1)基于VQVAE的运动量化模块,其将连续运动序列表示为离散潜在码\cite{van2017neural},2)模态不可知的Transformer编码器\cite{vaswani2017attention},其学习将模态感知驱动信号映射到关节空间,以及3)统一令牌Transformer(类GPT \cite{radford2019language})网络,用于以自回归方式预测量化的潜码索引。4)扩散运动解码器,其将运动标记作为输入,并将它们解码成具有高多样性的运动序列。在HumanML 3D \cite{Guo_2022_CVPR}和AIST++ \cite{li2021learn}基准测试上对该方法进行了测试,实验结果表明,该方法具有较好的性能.项目网址:\url {https:github.com/zixiangzhou916/UDE/

英文摘要:

Generating controllable and editable human motion sequences is a key challenge in 3D Avatar generation. It has been labor-intensive to generate and animate human motion for a long time until learning-based approaches have been developed and applied recently. However, these approaches are still task-specific or modality-specific\cite {ahuja2019language2pose}\cite{ghosh2021synthesis}\cite{ferreira2021learning}\cite{li2021ai}. In this paper, we propose ``UDE", the first unified driving engine that enables generating human motion sequences from natural language or audio sequences (see Fig.~\ref{fig:teaser}). Specifically, UDE consists of the following key components: 1) a motion quantization module based on VQVAE that represents continuous motion sequence as discrete latent code\cite{van2017neural}, 2) a modality-agnostic transformer encoder\cite{vaswani2017attention} that learns to map modality-aware driving signals to a joint space, and 3) a unified token transformer (GPT-like\cite{radford2019language}) network to predict the quantized latent code index in an auto-regressive manner. 4) a diffusion motion decoder that takes as input the motion tokens and decodes them into motion sequences with high diversity. We evaluate our method on HumanML3D\cite{Guo_2022_CVPR} and AIST++\cite{li2021learn} benchmarks, and the experiment results demonstrate our method achieves state-of-the-art performance. Project website: \url{https://github.com/zixiangzhou916/UDE/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑