arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1809.00263cs.CVcs.AIcs.LG
收藏

视频填充的随机动力学

Stochastic Dynamics for Video Infilling

University of Southern California(南加州大学);Nanyang Technological University(南洋理工大学);Columbia University(哥伦比亚大学)

Qiangeng Xu, Hanwang Zhang, Weiyue Wang, Peter N. Belhumeur, Ulrich Neumann

摘要:

在本文中,我们介绍了一个随机动态视频填充(SDVI)框架,以产生视频中的长间隔之间的帧。我们的任务不同于视频插值,其目的是产生过渡帧之间的每两帧的时间间隔很短,并增加时间分辨率。我们的任务,即视频填充,然而,目的是填充长时间间隔与合理的帧序列。我们的框架模型的填充作为一个受约束的随机生成过程,并从推断的分布顺序采样动态。SDVI由两部分组成:(1)双向约束传播模块,以保证帧之间的时空相干性,(2)随机采样过程,以从推断的分布生成动态。实验结果表明,SDVI可以生成清晰的、内容变化的帧序列。此外,生成的序列中的运动是真实的,并且能够从给定的开始帧平滑地转移到终端帧。我们的项目网站是https://xharlie.github.io/projects/project_sites/SDVI/video_results.html

英文摘要:

In this paper, we introduce a stochastic dynamics video infilling (SDVI) framework to generate frames between long intervals in a video. Our task differs from video interpolation which aims to produce transitional frames for a short interval between every two frames and increase the temporal resolution. Our task, namely video infilling, however, aims to infill long intervals with plausible frame sequences. Our framework models the infilling as a constrained stochastic generation process and sequentially samples dynamics from the inferred distribution. SDVI consists of two parts: (1) a bi-directional constraint propagation module to guarantee the spatial-temporal coherence among frames, (2) a stochastic sampling process to generate dynamics from the inferred distributions. Experimental results show that SDVI can generate clear frame sequences with varying contents. Moreover, motions in the generated sequence are realistic and able to transfer smoothly from the given start frame to the terminal frame. Our project site is https://xharlie.github.io/projects/project_sites/SDVI/video_results.html

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑