使用 Neural ODE 建模 StyleGAN 的潜空间动力学
Modelling Latent Dynamics of StyleGAN using Neural ODEs
AI总结：
本文用 Neural ODE 学习 StyleGAN 反演潜码的连续状态转移轨迹，实现无限帧插值与仅需编辑首帧的时间一致视频操控，并以更低计算达到最优性能。
摘要：
本文提出通过学习从 GAN 独立反演得到的潜码轨迹来建模视频动力学。通过将每个潜码视为运动粒子、将潜空间视为高维动态系统，整个序列被看作初始潜码连续轨迹的离散时间观测。因此，表示不同帧的潜码被重新表述为初始帧的状态转移，可用 neural ordinary differential equations（神经常微分方程）建模。学到的连续轨迹使我们能够执行无限帧插值和一致视频操控。后一任务被重新引入视频编辑，其优势在于只需将核心操作应用于第一帧，同时保持所有帧之间的时间一致性。大量实验表明，我们的方法以少得多的计算量取得了 state-of-the-art 性能。代码位于 https://github.com/weihaox/dynode_released。
英文摘要：
In this paper, we propose to model the video dynamics by learning the trajectory of independently inverted latent codes from GANs. The entire sequence is seen as discrete-time observations of a continuous trajectory of the initial latent code, by considering each latent code as a moving particle and the latent space as a high-dimensional dynamic system. The latent codes representing different frames are therefore reformulated as state transitions of the initial frame, which can be modeled by neural ordinary differential equations. The learned continuous trajectory allows us to perform infinite frame interpolation and consistent video manipulation. The latter task is reintroduced for video editing with the advantage of requiring the core operations to be applied to the first frame only while maintaining temporal consistency across all frames. Extensive experiments demonstrate that our method achieves state-of-the-art performance but with much less computation. Code is available at https://github.com/weihaox/dynode_released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
