arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.10909cs.CV
收藏

FrankenMotion: 部位级人类动作生成与组合

FrankenMotion: Part-level Human Motion Generation and Composition

Tübingen AI Center, University of Tübingen(图宾根人工智能中心,图宾根大学);Max Planck Institute for Informatics, Saarland Informatics Campus(马克斯·普朗克信息研究所,萨尔兰信息校区)

Chuqiao Li, Xianghui Xie, Yong Cao, Andreas Geiger, Gerard Pons-Moll

AI总结:

FrankenMotion通过原子性、时间感知的部位级动作标注,实现了对身体部位和原子动作的双重控制,首次在动作生成中引入细粒度部位级标注。

摘要:

从文本提示生成人类动作在近年来取得了显著进展。然而,现有方法主要依赖于序列级或动作级描述,由于缺乏细粒度、部位级的动作标注。这限制了它们对个体身体部位的可控性。在本工作中,我们构建了一个高质量的动作数据集,具有原子性、时间感知的部位级文本标注,利用大语言模型(LLMs)的推理能力。与之前的数据集不同,这些数据集要么提供同步的部位描述,固定时间段,要么仅依赖于全局序列标签,我们的数据集在细时间分辨率上捕捉到异步且语义不同的部位运动。基于此数据集,我们引入了一种基于扩散的部位感知动作生成框架,即FrankenMotion,其中每个身体部位由其自己的时间结构文本提示引导。据我们所知,这是第一个提供原子性、时间感知的部位级动作标注,并且有模型允许在空间(身体部位)和时间(原子动作)上进行动作生成的工作。实验表明,FrankenMotion在所有先前基线模型适应和重新训练后均表现更优,且我们的模型可以组合训练期间未见过的动作。我们的代码和数据集将在发表时公开。

英文摘要:

Human motion generation from text prompts has made remarkable progress in recent years. However, existing methods primarily rely on either sequence-level or action-level descriptions due to the absence of fine-grained, part-level motion annotations. This limits their controllability over individual body parts. In this work, we construct a high-quality motion dataset with atomic, temporally-aware part-level text annotations, leveraging the reasoning capabilities of large language models (LLMs). Unlike prior datasets that either provide synchronized part captions with fixed time segments or rely solely on global sequence labels, our dataset captures asynchronous and semantically distinct part movements at fine temporal resolution. Based on this dataset, we introduce a diffusion-based part-aware motion generation framework, namely FrankenMotion, where each body part is guided by its own temporally-structured textual prompt. This is, to our knowledge, the first work to provide atomic, temporally-aware part-level motion annotations and have a model that allows motion generation with both spatial (body part) and temporal (atomic action) control. Experiments demonstrate that FrankenMotion outperforms all previous baseline models adapted and retrained for our setting, and our model can compose motions unseen during training. Our code and dataset will be publicly available upon publication.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑