arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2003.00196cs.CVcs.AI
收藏

图像动画的一阶运动模型

First Order Motion Model for Image Animation

University of Trento(特伦托大学);Télécom Paris(巴黎高等电信学院);Institut polytechnique de Paris(巴黎理工学院);Snap Inc.(Snap公司);Fondazione Bruno Kessler(布鲁诺·凯斯勒基金会);Huawei Technologies Ireland(爱尔兰华为技术有限公司)

Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, Nicu Sebe

摘要:

图像动画包括生成视频序列,使得源图像中的对象根据驱动视频的运动而动画。我们的框架解决了这个问题,而不使用任何注释或有关特定对象的先验信息进行动画。一旦在一组描述相同类别对象(例如人脸,人体)的视频上进行训练,我们的方法就可以应用于该类的任何对象。为了实现这一点,我们解耦的外观和运动信息使用自监督制定。为了支持复杂的运动,我们使用了由一组学习的关键点及其局部仿射变换组成的表示。生成器网络对目标运动期间出现的遮挡进行建模,并将从源图像提取的外观和从驾驶视频导出的运动进行组合。我们的框架在不同的基准测试和各种对象类别上得分最高。我们的源代码是公开的。

英文摘要:

Image animation consists of generating a video sequence so that an object in a source image is animated according to the motion of a driving video. Our framework addresses this problem without using any annotation or prior information about the specific object to animate. Once trained on a set of videos depicting objects of the same category (e.g. faces, human bodies), our method can be applied to any object of this class. To achieve this, we decouple appearance and motion information using a self-supervised formulation. To support complex motions, we use a representation consisting of a set of learned keypoints along with their local affine transformations. A generator network models occlusions arising during target motions and combines the appearance extracted from the source image and the motion derived from the driving video. Our framework scores best on diverse benchmarks and on a variety of object categories. Our source code is publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑