arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2301.06052cs.CV
收藏

T2M-GPT:利用离散表示从文本描述生成人体运动

T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations

Jilin University(吉林大学);Shanghai Jiao Tong University(上海交通大学);Tencent AI Lab(腾讯人工智能实验室)

Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Shaoli Huang, Yong Zhang, Hongwei Zhao, Hongtao Lu, Xi Shen

AI总结:

本文提出T2M-GPT,结合VQ-VAE离散表示与GPT,通过简单训练策略在HumanML3D上以FID 0.116显著优于扩散方法,证明VQ-VAE在文本驱动人体运动生成中仍具竞争力。

摘要:

在这项工作中,我们研究了一个简单且必须了解的基于向量量化变分自编码器(VQ-VAE)和生成式预训练Transformer(GPT)的条件生成框架,用于从文本描述生成人体运动。我们表明,一个简单的基于CNN的VQ-VAE,配合常用的训练方法(EMA和Code Reset),能够让我们获得高质量的离散表示。对于GPT,我们在训练过程中引入了一种简单的损坏策略,以缓解训练与测试之间的差异。尽管方法简单,我们的T2M-GPT表现优于竞争性方法,包括最近的基于扩散的方法。例如,在目前最大的数据集HumanML3D上,我们在文本与生成运动之间的一致性(R-Precision)上取得了可比的性能,但FID为0.116,大幅优于MotionDiffuse的0.630。此外,我们对HumanML3D进行了分析,并观察到数据集规模是我们方法的一个限制因素。我们的工作表明,VQ-VAE仍然是人体运动生成的一种有竞争力的方法。

英文摘要:

In this work, we investigate a simple and must-known conditional generative framework based on Vector Quantised-Variational AutoEncoder (VQ-VAE) and Generative Pre-trained Transformer (GPT) for human motion generation from textural descriptions. We show that a simple CNN-based VQ-VAE with commonly used training recipes (EMA and Code Reset) allows us to obtain high-quality discrete representations. For GPT, we incorporate a simple corruption strategy during the training to alleviate training-testing discrepancy. Despite its simplicity, our T2M-GPT shows better performance than competitive approaches, including recent diffusion-based approaches. For example, on HumanML3D, which is currently the largest dataset, we achieve comparable performance on the consistency between text and generated motion (R-Precision), but with FID 0.116 largely outperforming MotionDiffuse of 0.630. Additionally, we conduct analyses on HumanML3D and observe that the dataset size is a limitation of our approach. Our work suggests that VQ-VAE still remains a competitive approach for human motion generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑