arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2411.08656cs.CV
收藏

MikuDance:利用混合运动动力学为角色艺术制作动画

MikuDance: Animating Character Art with Mixed Motion Dynamics

Wuhan University(武汉大学);StepFun;ByteDance(字节跳动)

Jiaxu Zhang, Xianfang Zeng, Xin Chen, Wei Zuo, Gang Yu, Zhigang Tu

AI总结:

提出基于扩散模型的MikuDance流水线,通过混合运动建模与混合控制扩散技术,解决角色艺术动画中高动态运动和参考引导不一致问题,实现高质量动画生成。

摘要:

我们提出了MikuDance,一个结合混合运动动力学的基于扩散模型的流水线,用于为风格化角色艺术制作动画。MikuDance包含两项关键技术:混合运动建模和混合控制扩散,以解决角色艺术动画中高动态运动和参考引导不一致的挑战。具体而言,提出了一种场景运动跟踪策略,在像素级空间中显式建模动态相机,实现统一的角色-场景运动建模。在此基础上,混合控制扩散隐式地将各种角色的比例和体型与运动引导对齐,允许对局部角色运动进行灵活控制。随后,引入了运动自适应归一化模块,以有效注入全局场景运动,为全面的角色艺术动画铺平道路。通过大量实验,我们证明了MikuDance在各种角色艺术和运动引导中的有效性和泛化性,始终如一地产生具有显著运动动力学的高质量动画。

英文摘要:

We propose MikuDance, a diffusion-based pipeline incorporating mixed motion dynamics to animate stylized character art. MikuDance consists of two key techniques: Mixed Motion Modeling and Mixed-Control Diffusion, to address the challenges of high-dynamic motion and reference-guidance misalignment in character art animation. Specifically, a Scene Motion Tracking strategy is presented to explicitly model the dynamic camera in pixel-wise space, enabling unified character-scene motion modeling. Building on this, the Mixed-Control Diffusion implicitly aligns the scale and body shape of diverse characters with motion guidance, allowing flexible control of local character motion. Subsequently, a Motion-Adaptive Normalization module is incorporated to effectively inject global scene motion, paving the way for comprehensive character art animation. Through extensive experiments, we demonstrate the effectiveness and generalizability of MikuDance across various character art and motion guidance, consistently producing high-quality animations with remarkable motion dynamics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑