arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.08462cs.CV
收藏

TriC-Motion: 基于三域因果建模的文本到动作生成

TriC-Motion: Tri-Domain Causal Modeling Grounded Text-to-Motion Generation

Inclusion AI, Ant Group, Beijing, China(蚂蚁集团Inclusion AI北京);Whiting School of Engineering, Johns Hopkins University, Baltimore, Maryland, USA(约翰霍普金斯大学工程学院)

Yiyang Cao, Yunze Deng, Ziyu Lin, Bin Feng, Xinggang Wang, Wenyu Liu, Dandan Zheng, Jingdong Chen

AI总结:

TriC-Motion通过整合空间-时间-频域建模与因果干预,提出一种基于扩散的文本到动作生成框架,实现高质量、连贯且与文本对齐的动作序列生成。

摘要:

文本到动作生成,计算机视觉领域中迅速发展的研究方向,旨在生成逼真且与文本对齐的动作序列。当前方法主要集中在空间-时间建模或独立频域分析,缺乏一个统一的框架,用于在空间、时间和频域之间进行联合优化。这种局限性阻碍了模型同时利用所有领域信息的能力,导致生成质量不佳。此外,在动作生成框架中,噪声引起的动作无关线索往往与对生成有积极贡献的特征交织在一起,从而导致动作失真。为了解决这些问题,我们提出了Tri-Domain Causal Text-to-Motion Generation(TriC-Motion),一种新的基于扩散的框架,整合了空间-时间-频域建模与因果干预。TriC-Motion包含三个核心建模模块,用于领域特定建模,即时间动作编码、空间拓扑建模和混合频域分析。在全面建模之后,一个Score引导的三域融合模块整合了三个领域中的有价值信息,同时确保时间一致性、空间拓扑、动作趋势和动态。此外,基于因果的反事实动作解构器被精心设计,以揭示动作无关线索,消除噪声,分离每个领域的真实建模贡献,从而实现更优的生成。广泛的实验结果验证了TriC-Motion相比最先进方法的优越性能,在HumanML3D数据集上取得了出色的R@1为0.612。这些结果展示了其生成高保真、连贯、多样且与文本对齐的动作序列的能力。代码可在:https://caoyiyang1105.github.io/TriC-Motion/上获取。

英文摘要:

Text-to-motion generation, a rapidly evolving field in computer vision, aims to produce realistic and text-aligned motion sequences. Current methods primarily focus on spatial-temporal modeling or independent frequency domain analysis, lacking a unified framework for joint optimization across spatial, temporal, and frequency domains. This limitation hinders the model's ability to leverage information from all domains simultaneously, leading to suboptimal generation quality. Additionally, in motion generation frameworks, motion-irrelevant cues caused by noise are often entangled with features that contribute positively to generation, thereby leading to motion distortion. To address these issues, we propose Tri-Domain Causal Text-to-Motion Generation (TriC-Motion), a novel diffusion-based framework integrating spatial-temporal-frequency-domain modeling with causal intervention. TriC-Motion includes three core modeling modules for domain-specific modeling, namely Temporal Motion Encoding, Spatial Topology Modeling, and Hybrid Frequency Analysis. After comprehensive modeling, a Score-guided Tri-domain Fusion module integrates valuable information from the triple domains, simultaneously ensuring temporal consistency, spatial topology, motion trends, and dynamics. Moreover, the Causality-based Counterfactual Motion Disentangler is meticulously designed to expose motion-irrelevant cues to eliminate noise, disentangling the real modeling contributions of each domain for superior generation. Extensive experimental results validate that TriC-Motion achieves superior performance compared to state-of-the-art methods, attaining an outstanding R@1 of 0.612 on the HumanML3D dataset. These results demonstrate its capability to generate high-fidelity, coherent, diverse, and text-aligned motion sequences. Code is available at: https://caoyiyang1105.github.io/TriC-Motion/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑