arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.03311cs.CVcs.LG
收藏

利用百万级人体动作扩展大动作模型

Scaling Large Motion Models with Million-Level Human Motions

Renmin University of China(中国人民大学);Southeast University(东南大学);BeingBeyond

Ye Wang, Sipeng Zheng, Bin Cao, Qianshan Wei, Weishuai Zeng, Qin Jin, Zongqing Lu

AI总结:

为解决缺乏海量高质量数据的问题,本研究提出了百万级动作数据集MotionLib,并训练大动作模型,同时引入创新的动作编码方法Motionbook,首次揭示了数据和模型规模扩展对动作生成的重要性。

摘要:

受LLMs近期成功的启发,人体动作理解领域已日益转向开发大动作模型。尽管取得了一些进展,但当前的努力仍远未实现真正的通用模型,主要原因是缺乏海量高质量数据。为了解决这一差距,我们提出了MotionLib,这是首个用于动作生成的百万级数据集,其规模比现有同类数据集大至少15倍,并富含分层文本描述。使用MotionLib,我们训练了一个名为\projname的大动作模型,展示了在包括未见动作在内的广泛人体活动中的稳健性能。通过系统调查,我们首次强调了扩展数据和模型规模对推进动作生成的重要性,并提供了实现这一目标的关键见解。为了更好地整合动作模态,我们提出了Motionbook,这是一种创新的动作编码方法,包括(1)一种紧凑且无损的特征来表示动作;(2)一种新颖的2D无查找动作分词器,在扩展码本容量的同时保留细粒度的动作细节,显著增强了动作标记的表示能力。我们相信这项工作为未来开发更通用、更强大的动作生成模型奠定了基础。更多详情,请访问https://beingbeyond.github.io/Being-M0/。

英文摘要:

Inspired by the recent success of LLMs, the field of human motion understanding has increasingly shifted toward developing large motion models. Despite some progress, current efforts remain far from achieving truly generalist models, primarily due to the lack of massive high-quality data. To address this gap, we present MotionLib, the first million-level dataset for motion generation, which is at least 15$\times$ larger than existing counterparts and enriched with hierarchical text descriptions. Using MotionLib, we train a large motion model named \projname, demonstrating robust performance across a wide range of human activities, including unseen ones. Through systematic investigation, for the first time, we highlight the importance of scaling both data and model size for advancing motion generation, along with key insights to achieve this goal. To better integrate the motion modality, we propose Motionbook, an innovative motion encoding approach including (1) a compact yet lossless feature to represent motions; (2) a novel 2D lookup-free motion tokenizer that preserves fine-grained motion details while expanding codebook capacity, significantly enhancing the representational power of motion tokens. We believe this work lays the groundwork for developing more versatile and powerful motion generation models in the future. For further details, visit https://beingbeyond.github.io/Being-M0/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑