AI 中文总结
MoRAE针对文本到动作生成中直接采用图像生成RAE范式的失败，解决了动作空间的两个特有瓶颈，使Flow-Matching DiT实现了最先进的文本到动作生成性能。
AI 中文摘要
文本到动作生成必须生成语义正确、时间连贯且物理合理的动作。一种自然的方法是先将动作数据投影到结构化语义空间，再在该空间内训练生成模型。这种范式在图像生成中通过表示自编码器（RAE）取得了巨大成功，其中冻结的自监督编码器为扩散模型或流模型提供语义特征供其学习。然而，将该范式直接应用于动作空间、以Motion-JEPA作为冻结编码器的尝试却以失败告终。我们从几何角度分析了这一失败，确定了两个动作特有的瓶颈：（1）JEPA特征空间在频谱上病态，导致高斯到数据的传输不稳定；（2）即便频谱状况良好，流残差也倾向于与解码器敏感方向对齐，微小的隐变量误差在解码后会被放大为大幅动作伪影。基于这些见解，我们提出了MoRAE。MoRAE分别解决这两个瓶颈：紧凑瓶颈提炼结构化JEPA表示，同时去除弱且冗余的方向，使隐变量频谱进入传输稳定状态；动作耦合训练随后将保留的隐变量几何与解码器对齐，使特征流误差在解码后代价更低。凭借这种流友好型隐变量，标准非自回归Flow-Matching DiT实现了最先进的性能。
英文摘要
Text-to-motion generation must produce motions that are semantically correct, temporally coherent, and physically plausible. A natural approach is to first project motion data into a structured semantic space and then train a generative model within that space. Such a paradigm has been highly successful in image generation through Representation Autoencoders (RAEs), where a frozen self-supervised encoder provides semantic features for diffusion or flow models to learn from. However, direct transfer of such a paradigm to motion space using Motion-JEPA as the frozen encoder fails dramatically. We diagnose this failure geometrically and identify two motion-specific bottlenecks: (1) the JEPA feature space is spectrally ill-conditioned, making the Gaussian-to-data transport unstable; and (2) even with a well-conditioned spectrum, flow residuals tend to align with decoder-sensitive directions, where small latent errors are amplified into large motion artifacts after decoding. Based on these insights, we propose MoRAE. MoRAE addresses the two bottlenecks separately. A compact bottleneck distills the structured JEPA representation while removing weak and redundant directions, bringing the latent spectrum into a transport-stable regime. Motion-coupled training then aligns the retained latent geometry with the decoder, making characteristic flow errors less costly after decoding. With this flow-friendly latent, a standard non-autoregressive Flow-Matching DiT achieves state-of-the-art performance.
