arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2304.05684cs.CV
收藏

InterGen:复杂交互下基于扩散的多人体运动生成

InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions

ShanghaiTech University(上海科技大学)

Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, Lan Xu

AI总结:

本文提出 InterGen 扩散模型及 InterHuman 数据集,通过共享权重的双 Transformer 去噪器、互注意力机制和空间关系正则化,实现文本引导的高质量双人交互运动生成。

摘要:

近年来,我们在利用扩散模型生成逼真人体运动方面取得了巨大进展,但这些方法在很大程度上忽略了多人之间的交互。本文提出了 InterGen,一种有效的基于扩散的方法,将人与人之间的交互纳入运动扩散过程,使非专业用户仅通过文本引导即可定制高质量的双人交互运动。我们首先贡献了一个多模态数据集 InterHuman,包含约 1.07 亿帧多样化的双人交互数据,具有精确的骨骼运动和 23,337 条自然语言描述。在算法方面,我们针对双人交互场景精心设计了运动扩散模型。为处理交互过程中人体身份的对称性,我们提出了两个协作的基于 Transformer 的去噪器,它们显式共享权重,并通过互注意力机制进一步连接两个去噪过程。然后,我们在交互扩散模型中提出了一种新颖的运动输入表示,显式刻画了世界坐标系下两位表演者之间的全局关系。我们进一步引入了两个新颖的正则化项来编码空间关系,并在交互扩散模型训练过程中配备了相应的阻尼方案。大量实验验证了 InterGen 的有效性和泛化能力。值得注意的是,与以往方法相比,它能够生成更加多样且引人注目的双人运动,并支持多种面向人类交互的下游应用。

英文摘要:

We have recently seen tremendous progress in diffusion advances for generating realistic human motions. Yet, they largely disregard the multi-human interactions. In this paper, we present InterGen, an effective diffusion-based approach that incorporates human-to-human interactions into the motion diffusion process, which enables layman users to customize high-quality two-person interaction motions, with only text guidance. We first contribute a multimodal dataset, named InterHuman. It consists of about 107M frames for diverse two-person interactions, with accurate skeletal motions and 23,337 natural language descriptions. For the algorithm side, we carefully tailor the motion diffusion model to our two-person interaction setting. To handle the symmetry of human identities during interactions, we propose two cooperative transformer-based denoisers that explicitly share weights, with a mutual attention mechanism to further connect the two denoising processes. Then, we propose a novel representation for motion input in our interaction diffusion model, which explicitly formulates the global relations between the two performers in the world frame. We further introduce two novel regularization terms to encode spatial relations, equipped with a corresponding damping scheme during the training of our interaction diffusion model. Extensive experiments validate the effectiveness and generalizability of InterGen. Notably, it can generate more diverse and compelling two-person motions than previous methods and enables various downstream applications for human interactions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑