UniMotion: 一种用于运动-文本-视觉理解和生成的统一框架
UniMotion: A Unified Framework for Motion-Text-Vision Understanding and Generation
AI总结：
UniMotion首次提出统一框架，同时处理人类运动、自然语言和RGB图像的理解与生成，通过连续模态处理和自监督预训练解决传统模型的局限性。
摘要：
我们提出了UniMotion，据我们所知，这是首个统一框架，能够在单一架构中同时理解和生成人类运动、自然语言和RGB图像。现有统一模型仅处理受限模态子集（例如运动-文本或静态姿态-图像），并主要依赖离散标记化，这引入了量化误差并破坏了时间连续性。UniMotion通过核心原则克服了这些限制：将运动视为与RGB同等重要的连续模态。一种新颖的跨模态对齐运动变分自编码器（CMA-VAE）和对称双路径嵌入器在共享LLM主干中构建并行连续路径。为在推理时无需图像注入视觉-语义先验到运动表示，我们提出双后验KL对齐（DPA），将视觉融合编码器的丰富后验蒸馏到运动-only编码器中。为解决冷启动问题——即文本监督单独太稀疏无法校准新引入的运动路径——我们进一步提出潜在重构对齐（LRA），一种自监督预训练策略，利用密集运动潜在作为无歧义条件，共同校准嵌入器、主干和流头，为所有下游任务建立稳定的运动感知基础。UniMotion在七个任务上实现了最先进的性能，涵盖任何到任何的理解、生成和编辑，尤其在跨模态组合任务上表现突出。
英文摘要：
We present UniMotion, to our knowledge the first unified framework for simultaneous understanding and generation of human motion, natural language, and RGB images within a single architecture. Existing unified models handle only restricted modality subsets (e.g., Motion-Text or static Pose-Image) and predominantly rely on discrete tokenization, which introduces quantization errors and disrupts temporal continuity. UniMotion overcomes both limitations through a core principle: treating motion as a first-class continuous modality on equal footing with RGB. A novel Cross-Modal Aligned Motion VAE (CMA-VAE) and symmetric dual-path embedders construct parallel continuous pathways for Motion and RGB within a shared LLM backbone. To inject visual-semantic priors into motion representations without requiring images at inference, we propose Dual-Posterior KL Alignment (DPA), which distills a vision-fused encoder's richer posterior into the motion-only encoder. To address the cold-start problem -- where text supervision alone is too sparse to calibrate the newly introduced motion pathway -- we further propose Latent Reconstruction Alignment (LRA), a self-supervised pre-training strategy that uses dense motion latents as unambiguous conditions to co-calibrate the embedder, backbone, and flow head, establishing a stable motion-aware foundation for all downstream tasks. UniMotion achieves state-of-the-art performance across seven tasks spanning any-to-any understanding, generation, and editing among the three modalities, with especially strong advantages on cross-modal compositional tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
