WAIT: 基于GAN的动画到插画视频转换的特征扭曲方法
WAIT: Feature Warping for Animation to Illustration video Translation using GANs
AI总结：
针对使用无序插画集合给动画视频风格化的新问题，提出带特征扭曲层的GAN生成器网络，无需额外光流等网络即可保证时间连贯性，在三个数据集上验证了有效性。
摘要：
本文探索了视频到视频转换的一个新领域。受改编自儿童插画书的动画电影的启发，我们旨在用原始插画的风格对这些视频进行风格化处理。当前最先进的视频到视频转换模型依赖于一段视频序列或单张风格图像来对输入视频进行风格化。我们提出了一个新的视频风格化问题，即使用无序图像集合进行风格化。这一任务具有挑战性，原因有二：其一，我们不具备视频序列所拥有的时间一致性优势；其二，与使用单张图像相比，从一组无序图像中为视频帧获取一致的风格更为困难。大多数视频到视频转换方法都建立在图像到图像转换模型的基础上，并集成光流或时间预测器等额外网络来捕捉时间关系。这些额外网络使模型训练和推理变得复杂，还会拖慢处理速度。为确保视频到视频风格迁移中的时间连贯性，我们提出了一种带有特征扭曲层的新型生成器网络，克服了以往方法的局限性。我们在三个数据集上从定性和定量两方面验证了所提方法的有效性。代码和预训练模型可在https://github.com/giddyyupp/wait获取。
英文摘要：
In this paper, we explore a new domain for video-to-video translation. Motivated by the availability of animation movies that are adopted from illustrated books for children, we aim to stylize these videos with the style of the original illustrations. Current state-of-the-art video-to-video translation models rely on having a video sequence or a single style image to stylize an input video. We introduce a new problem for video stylizing where an unordered set of images are used. This is a challenging task for two reasons: i) we do not have the advantage of temporal consistency as in video sequences; ii) it is more difficult to obtain consistent styles for video frames from a set of unordered images compared to using a single image. Most of the video-to-video translation methods are built on an image-to-image translation model, and integrate additional networks such as optical flow, or temporal predictors to capture temporal relations. These additional networks make the model training and inference complicated and slow down the process. To ensure temporal coherency in video-to-video style transfer, we propose a new generator network with feature warping layers which overcomes the limitations of the previous methods. We show the effectiveness of our method on three datasets both qualitatively and quantitatively. Code and pretrained models are available at https://github.com/giddyyupp/wait.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
