我们永不脱离 Style:通过潜空间子空间分解实现运动解耦
We never go out of Style: Motion Disentanglement by Subspace Decomposition of Latent Space
AI总结:
针对视频中多成分复杂运动的解耦问题,本文利用基于 style 的 GAN 潜空间发现语义运动子空间,仅需约10个真值视频即可支持运动编辑与选择性迁移。
摘要:
现实世界中的物体会执行涉及多个独立运动成分的复杂运动。例如,一个人在说话时会不断改变其表情、头部和身体姿态。在本工作中,我们提出了一种利用预训练图像 GAN 模型分解视频中运动的新方法。我们在广泛使用的基于 style 的 GAN 模型的潜空间中发现了具有语义意义、并控制单一可解释运动成分的解耦运动子空间。所提出的方法仅需少量(约10个)真值视频序列即可获得此类子空间。我们在人脸和汽车数据集上从定量和定性两方面广泛评估了运动子空间的解耦特性。此外,我们展示了多个下游任务的结果,例如运动编辑和选择性运动迁移,例如在不进行针对性训练的情况下仅迁移面部表情。
英文摘要:
Real-world objects perform complex motions that involve multiple independent motion components. For example, while talking, a person continuously changes their expressions, head, and body pose. In this work, we propose a novel method to decompose motion in videos by using a pretrained image GAN model. We discover disentangled motion subspaces in the latent space of widely used style-based GAN models that are semantically meaningful and control a single explainable motion component. The proposed method uses only a few $(\approx10)$ ground truth video sequences to obtain such subspaces. We extensively evaluate the disentanglement properties of motion subspaces on face and car datasets, quantitatively and qualitatively. Further, we present results for multiple downstream tasks such as motion editing, and selective motion transfer, e.g. transferring only facial expressions without training for it.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。