arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.15975cs.CV
收藏

UMO:统一上下文学习解锁运动基础模型先验

UMO: Unified In-Context Learning Unlocks Motion Foundation Model Priors

Brown University(布朗大学);Massachusetts Institute of Technology(麻省理工学院);Meta Reality Lab(Meta现实实验室);Max-Planck Institute for Intelligent Systems(智能系统马克斯·普朗克研究所);University of Hong Kong(香港大学)

Xiaoyan Cong, Zekun Li, Zhiyang Dou, Hongyu Li, Omid Taheri, Chuan Guo, Abhay Mittal, Sizhe An, Taku Komura, Wojciech Matusik, Michael J. Black, Srinath Sridhar

AI总结:

UMO通过统一框架解锁运动基础模型先验,支持多种跨模态和上下文生成任务,提升文本到运动合成性能。

摘要:

大规模基础模型(LFMs)通过学习大量3D人体运动数据集和配对文本描述,已取得文本到运动生成的显著进展。然而,如何有效利用单用途运动LFMs在更多跨模态和上下文运动生成下游任务中仍不明确。本文提出UMO,通过统一框架将不同下游任务转化为原子级帧操作,实现上下文适应以解锁预训练DiT运动LFMs的生成先验。UMO引入三个可学习的帧级元操作嵌入以指定帧意图,并采用轻量级时间融合注入上下文提示,运行时开销极低。实验表明,UMO在多种基准上优于任务特定和无训练基线,尽管使用单一统一模型。代码和模型将公开。项目页面:https://oliver-cong02.github.io/UMO.github.io/

英文摘要:

Large-scale foundation models (LFMs) have recently made impressive progress in text-to-motion generation by learning strong generative priors from massive 3D human motion datasets and paired text descriptions. However, how to effectively and efficiently leverage such single-purpose motion LFMs, i.e., text-to-motion synthesis, in more diverse cross-modal and in-context motion generation downstream tasks remains largely unclear. Prior work typically adapts pretrained generative priors to individual downstream tasks in a task-specific manner. In contrast, our goal is to unlock such priors to support a broad spectrum of downstream motion generation tasks within a single unified framework. To bridge this gap, we present UMO, a simple yet general unified formulation that casts diverse downstream tasks into compositions of atomic per-frame operations, enabling in-context adaptation to unlock the generative priors of pretrained DiT-based motion LFMs. Specifically, UMO introduces three learnable frame-level meta-operation embeddings to specify per-frame intent and employs lightweight temporal fusion to inject in-context cues into the pretrained backbone, with negligible runtime overhead compared to the base model. With this design, UMO finetunes the pretrained model, originally limited to text-to-motion generation, to support diverse previously unsupported tasks, including temporal inpainting, text-guided motion editing, text-serialized geometric constraints, and multi-identity reaction generation. Experiments demonstrate that UMO consistently outperforms task-specific and training-free baselines across a wide range of benchmarks, despite using a single unified model. Code and model will be publicly available. Project Page: https://oliver-cong02.github.io/UMO.github.io/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑