Dual-MTGAN：用于图像到视频合成的随机和确定性运动传输
Dual-MTGAN: Stochastic and Deterministic Motion Transfer for Image-to-Video Synthesis
摘要：
在计算机视觉中，生成具有内容和运动变化的视频是一项具有挑战性的任务。虽然GAN的最新发展允许从潜在表示生成视频，但是生成具有感兴趣的运动模式的特定内容的视频并不容易。在本文中，我们提出了双运动传输GAN（Dual-MT GAN），它以图像和视频数据作为输入，同时学习解纠缠的内容和运动表示。我们的Dual-MTGAN能够执行确定性运动传输和随机运动生成。基于给定的图像，前者保留输入内容并转移从另一个视频序列中观察到的运动模式，而后者直接基于输入图像产生具有合理但多样的运动模式的视频。所提出的模型以端到端的方式进行训练，而不需要利用预定义的运动特征，如姿势或面部标志。我们的定量和定性结果将证实我们的模型在解决这种条件图像到视频任务的有效性和鲁棒性。
英文摘要：
Generating videos with content and motion variations is a challenging task in computer vision. While the recent development of GAN allows video generation from latent representations, it is not easy to produce videos with particular content of motion patterns of interest. In this paper, we propose Dual Motion Transfer GAN (Dual-MTGAN), which takes image and video data as inputs while learning disentangled content and motion representations. Our Dual-MTGAN is able to perform deterministic motion transfer and stochastic motion generation. Based on a given image, the former preserves the input content and transfers motion patterns observed from another video sequence, and the latter directly produces videos with plausible yet diverse motion patterns based on the input image. The proposed model is trained in an end-to-end manner, without the need to utilize pre-defined motion features like pose or facial landmarks. Our quantitative and qualitative results would confirm the effectiveness and robustness of our model in addressing such conditioned image-to-video tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
