时间残差雅可比矩阵用于无绑定运动迁移
Temporal Residual Jacobians For Rig-free Motion Transfer
AI总结:
本文提出时间残差雅可比矩阵表示,通过两个耦合神经网络联合预测空间与时间变化并积分,实现无需绑定和关键帧的数据驱动运动迁移,在多样网格上生成逼真动画,优于现有方法。
摘要:
我们引入时间残差雅可比矩阵作为一种新颖的表示,以实现数据驱动的运动迁移。我们的方法不假设能访问任何绑定或中间形状关键帧,能够生成几何和时间上一致的运动,并可用于迁移长运动序列。我们方法的核心是两个耦合的神经网络,它们分别预测局部几何和时间变化,随后在空间和时间上进行积分,以生成最终的动画网格。这两个网络联合训练,在产生空间和时间信号方面互补,并直接以三维位置信息进行监督。在推理阶段,在没有关键帧的情况下,我们的方法本质上解决了一个运动外推问题。我们在多样化的网格(合成和扫描形状)上测试了我们的设置,以展示其在生成未见身体形状上逼真且自然的动画方面优于最先进的替代方法。补充视频和代码可在 https://temporaljacobians.github.io/ 获取。
英文摘要:
We introduce Temporal Residual Jacobians as a novel representation to enable data-driven motion transfer. Our approach does not assume access to any rigging or intermediate shape keyframes, produces geometrically and temporally consistent motions, and can be used to transfer long motion sequences. Central to our approach are two coupled neural networks that individually predict local geometric and temporal changes that are subsequently integrated, spatially and temporally, to produce the final animated meshes. The two networks are jointly trained, complement each other in producing spatial and temporal signals, and are supervised directly with 3D positional information. During inference, in the absence of keyframes, our method essentially solves a motion extrapolation problem. We test our setup on diverse meshes (synthetic and scanned shapes) to demonstrate its superiority in generating realistic and natural-looking animations on unseen body shapes against SoTA alternatives. Supplemental video and code are available at https://temporaljacobians.github.io/ .
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。