arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.02791cs.CVcs.AI
收藏

基于潜在一致性训练的高效文本驱动运动生成

Efficient Text-driven Motion Generation via Latent Consistency Training

Tongji university(同济大学)

Mengxian Hu, Minghao Zhu, Xun Zhou, Qingqing Yan, Shu Li, Chengju Liu, Qijun Chen

AI总结:

针对文本驱动人体运动生成中扩散模型推理效率低的问题,提出MLCT框架,通过运动自动编码器、无分类器引导和聚类引导模块实现少步/单步推理,训练成本更低且性能可与最先进模型媲美。

摘要:

基于扩散策略的文本驱动人体运动生成为人机交互中的多模态应用奠定了可靠基础。然而,现有进展面临显著的效率挑战,因为推理阶段迭代求解非线性反向扩散轨迹会产生大量计算开销。为此,我们提出运动潜在一致性训练框架(MLCT),该框架在训练阶段从原始数据中预计算反向扩散轨迹,并在推理阶段通过自一致性约束实现少步或单步推理。具体而言,首先提出带量化约束的运动自动编码器,为运动扩散过程构建简洁且有界的解分布。随后,通过附加的无条件损失函数构建无分类器引导格式,以在训练阶段完成条件扩散轨迹的预计算。最后,基于K近邻算法开发聚类引导模块,用于自一致性约束的链式传导优化机制,以较小的查询成本提供解分布的额外参考。通过结合这些增强措施,我们在非像素模态和潜在表示空间中实现了稳定且一致的训练。基准实验表明,我们的方法显著优于传统的一致性蒸馏方法,降低了训练成本,并增强了一致性模型,使其在较低推理成本下性能可与最先进模型媲美。

英文摘要:

Text-driven human motion generation based on diffusion strategies establishes a reliable foundation for multimodal applications in human-computer interactions. However, existing advances face significant efficiency challenges due to the substantial computational overhead of iteratively solving for nonlinear reverse diffusion trajectories during the inference phase. To this end, we propose the motion latent consistency training framework (MLCT), which precomputes reverse diffusion trajectories from raw data in the training phase and enables few-step or single-step inference via self-consistency constraints in the inference phase. Specifically, a motion autoencoder with quantization constraints is first proposed for constructing concise and bounded solution distributions for motion diffusion processes. Subsequently, a classifier-free guidance format is constructed via an additional unconditional loss function to accomplish the precomputation of conditional diffusion trajectories in the training phase. Finally, a clustering guidance module based on the K-nearest-neighbor algorithm is developed for the chain-conduction optimization mechanism of self-consistency constraints, which provides additional references of solution distributions at a small query cost. By combining these enhancements, we achieve stable and consistency training in non-pixel modality and latent representation spaces. Benchmark experiments demonstrate that our method significantly outperforms traditional consistency distillation methods with reduced training cost and enhances the consistency model to perform comparably to state-of-the-art models with lower inference costs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑