arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.03485cs.CVcs.GR
收藏

LGTM:局部到全局的文本驱动人体运动扩散模型

LGTM: Local-to-Global Text-Driven Human Motion Diffusion Model

Shenzhen University(深圳大学);Kuaishou Technology(快手科技);ByteDance Inc.(字节跳动公司)

Haowen Sun, Ruikun Zheng, Haibin Huang, Chongyang Ma, Hui Huang, Ruizhen Hu

AI总结:

LGTM提出局部到全局的文本驱动人体运动扩散模型,通过LLM分解描述、分部位编码和注意力全身优化,显著提升文本到运动生成的局部准确性与语义对齐。

摘要:

本文提出了LGTM,一种用于文本到运动生成的新型局部到全局流程。LGTM采用基于扩散的架构,旨在解决计算机动画中将文本描述准确转换为语义连贯的人体运动这一挑战。具体而言,传统方法常常面临语义不一致的问题,尤其是在将特定运动与正确的身体部位对齐方面。为解决这一问题,我们提出了一种两阶段流程:首先利用大语言模型(LLMs)将全局运动描述分解为针对特定身体部位的叙述,然后由独立的身体部位运动编码器处理这些叙述,以确保精确的局部语义对齐。最后,基于注意力的全身优化器对运动生成结果进行细化,并保证整体连贯性。实验表明,LGTM在生成局部准确、语义对齐的人体运动方面取得了显著改进,标志着文本到运动应用的重要进展。本文的代码和数据可在https://github.com/L-Sun/LGTM获取。

英文摘要:

In this paper, we introduce LGTM, a novel Local-to-Global pipeline for Text-to-Motion generation. LGTM utilizes a diffusion-based architecture and aims to address the challenge of accurately translating textual descriptions into semantically coherent human motion in computer animation. Specifically, traditional methods often struggle with semantic discrepancies, particularly in aligning specific motions to the correct body parts. To address this issue, we propose a two-stage pipeline to overcome this challenge: it first employs large language models (LLMs) to decompose global motion descriptions into part-specific narratives, which are then processed by independent body-part motion encoders to ensure precise local semantic alignment. Finally, an attention-based full-body optimizer refines the motion generation results and guarantees the overall coherence. Our experiments demonstrate that LGTM gains significant improvements in generating locally accurate, semantically-aligned human motion, marking a notable advancement in text-to-motion applications. Code and data for this paper are available at https://github.com/L-Sun/LGTM

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑