arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.18512cs.CV
收藏

ParCo:部位协调的文本到动作合成

ParCo: Part-Coordinating Text-to-Motion Synthesis

Tsinghua University(清华大学);Peking University Shenzhen Graduate School(北京大学深圳研究生院)

Qiran Zou, Shangyuan Yuan, Shian Du, Yu Wang, Chang Liu, Yi Xu, Jie Chen, Xiangyang Ji

AI总结:

本文提出 ParCo 框架,通过将全身动作离散化为部位动作并采用轻量级生成器与部位协调模块,实现协调且细粒度的文本到动作合成,在 HumanML3D 和 KIT-ML 基准上以经济计算量取得优越性能。

摘要:

我们研究一项具有挑战性的任务:文本到动作合成,旨在生成符合文本描述并展现协调运动的动作。当前基于部位的方法将部位划分引入动作合成过程以实现更细粒度的生成。然而,这些方法面临不同部位动作之间缺乏协调以及网络难以理解部位概念等挑战。此外,引入更细粒度的部位概念会带来计算复杂度的挑战。本文提出部位协调的文本到动作合成(ParCo),具备增强的部位动作理解能力和不同部位动作生成器之间的通信能力,确保协调且细粒度的动作合成。具体而言,我们将全身动作离散化为多个部位动作,以建立不同部位的先验概念。随后,我们采用多个轻量级生成器来合成不同部位动作,并通过我们的部位协调模块进行协调。我们的方法在 HumanML3D 和 KIT-ML 等通用基准上以经济的计算量展现出优越性能,为其有效性提供了充分证据。代码可在 https://github.com/qrzou/ParCo 获取。

英文摘要:

We study a challenging task: text-to-motion synthesis, aiming to generate motions that align with textual descriptions and exhibit coordinated movements. Currently, the part-based methods introduce part partition into the motion synthesis process to achieve finer-grained generation. However, these methods encounter challenges such as the lack of coordination between different part motions and difficulties for networks to understand part concepts. Moreover, introducing finer-grained part concepts poses computational complexity challenges. In this paper, we propose Part-Coordinating Text-to-Motion Synthesis (ParCo), endowed with enhanced capabilities for understanding part motions and communication among different part motion generators, ensuring a coordinated and fined-grained motion synthesis. Specifically, we discretize whole-body motion into multiple part motions to establish the prior concept of different parts. Afterward, we employ multiple lightweight generators designed to synthesize different part motions and coordinate them through our part coordination module. Our approach demonstrates superior performance on common benchmarks with economic computations, including HumanML3D and KIT-ML, providing substantial evidence of its effectiveness. Code is available at https://github.com/qrzou/ParCo .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑