arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.19227cs.CV
收藏

通过扩散基离散运动分词器弥合语义与运动条件

Bridging Semantic and Kinematic Conditions with Diffusion-based Discrete Motion Tokenizer

S-Lab, Nanyang Technological University(南洋理工大学S实验室);The Chinese University of Hong Kong(香港中文大学)

Chenyang Gu, Mingyuan Zhang, Haozhe Xie, Zhongang Cai, Lei Yang, Ziwei Liu

AI总结:

本文提出三阶段框架,结合连续扩散模型与离散分词生成器,通过MoTok分词器实现语义与运动控制的结合,提升运动生成的可控性与保真度。

摘要:

先前的运动生成主要遵循两种范式:擅长运动控制的连续扩散模型,以及擅长语义条件的离散分词生成器。为结合二者优势,我们提出包含条件特征提取(感知)、离散分词生成(规划)和扩散基运动合成(控制)的三阶段框架。核心是MoTok,一种基于扩散的离散运动分词器,通过将运动恢复委托给扩散解码器,实现了语义抽象与精细重建的解耦,从而生成紧凑的单层分词同时保持运动保真度。对于运动学条件,粗约束在规划期间指导分词生成,而细粒度约束通过扩散优化在控制期间强制执行。此设计防止运动学细节干扰语义分词规划。在HumanML3D上,我们的方法在仅使用六分之一分词的情况下,显著提升了可控性和保真度,将轨迹误差从0.72厘米降至0.08厘米,FID从0.083降至0.029。与以往方法不同,我们的方法在更强的运动学约束下仍能提升保真度,将FID从0.033降至0.014。

英文摘要:

Prior motion generation largely follows two paradigms: continuous diffusion models that excel at kinematic control, and discrete token-based generators that are effective for semantic conditioning. To combine their strengths, we propose a three-stage framework comprising condition feature extraction (Perception), discrete token generation (Planning), and diffusion-based motion synthesis (Control). Central to this framework is MoTok, a diffusion-based discrete motion tokenizer that decouples semantic abstraction from fine-grained reconstruction by delegating motion recovery to a diffusion decoder, enabling compact single-layer tokens while preserving motion fidelity. For kinematic conditions, coarse constraints guide token generation during planning, while fine-grained constraints are enforced during control through diffusion-based optimization. This design prevents kinematic details from disrupting semantic token planning. On HumanML3D, our method significantly improves controllability and fidelity over MaskControl while using only one-sixth of the tokens, reducing trajectory error from 0.72 cm to 0.08 cm and FID from 0.083 to 0.029. Unlike prior methods that degrade under stronger kinematic constraints, ours improves fidelity, reducing FID from 0.033 to 0.014.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑