VidStyleODE：基于StyleGAN与神经常微分方程的解耦视频编辑
VidStyleODE: Disentangled Video Editing via StyleGAN and NeuralODEs
AI总结：
本文提出VidStyleODE，将视频内容编码到StyleGAN潜在空间并用神经常微分方程建模运动动态，实现外观与运动的解耦，支持文本引导编辑、运动操控、图像动画及视频插值与外推。
摘要：
我们提出VidStyleODE，这是一种基于StyleGAN和神经常微分方程（Neural-ODE）的时空连续解耦视频表示。有效遍历生成对抗网络（GAN）学习到的潜在空间已成为近年来图像编辑突破的基础。然而，此类进展在视频领域的适用性一直受到在GAN潜在空间中表示和控制视频的困难的阻碍。特别是，视频由内容（即外观）和复杂的运动分量组成，需要一种特殊的机制来解耦和控制。为了实现这一目标，VidStyleODE将视频内容编码在预训练的StyleGAN的$\mathcal{W}_+$空间中，并利用潜在ODE分量来概括输入视频的时空动态。我们新颖的连续视频生成过程随后将二者结合，以生成具有不同帧率的高质量且时间一致的视频。我们表明，所提出的方法能够在真实视频上实现多种应用：文本引导的外观操控、运动操控、图像动画以及视频插值和外推。项目网站：https://cyberiada.github.io/VidStyleODE
英文摘要：
We propose $\textbf{VidStyleODE}$, a spatiotemporally continuous disentangled $\textbf{Vid}$eo representation based upon $\textbf{Style}$GAN and Neural-$\textbf{ODE}$s. Effective traversal of the latent space learned by Generative Adversarial Networks (GANs) has been the basis for recent breakthroughs in image editing. However, the applicability of such advancements to the video domain has been hindered by the difficulty of representing and controlling videos in the latent space of GANs. In particular, videos are composed of content (i.e., appearance) and complex motion components that require a special mechanism to disentangle and control. To achieve this, VidStyleODE encodes the video content in a pre-trained StyleGAN $\mathcal{W}_+$ space and benefits from a latent ODE component to summarize the spatiotemporal dynamics of the input video. Our novel continuous video generation process then combines the two to generate high-quality and temporally consistent videos with varying frame rates. We show that our proposed method enables a variety of applications on real videos: text-guided appearance manipulation, motion manipulation, image animation, and video interpolation and extrapolation. Project website: https://cyberiada.github.io/VidStyleODE
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
