arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1906.10101cs.CVcs.AI
收藏

LMVP:具有泄漏运动信息的视频预测器

LMVP: Video Predictor with Leaked Motion Information

Duke University(杜克大学);Tsinghua University(清华大学);JP Morgan & Chase(摩根大通)

Dong Wang, Yitong Li, Wei Cao, Liqun Chen, Qi Wei, Lawrence Carin

摘要:

我们提出了一个泄漏的运动视频预测器(LMVP)预测未来的帧捕获的空间和时间的依赖性,从给定的输入。该运动是由一个新提出的组件,运动引导器,它扮演着学习者和教师的角色。具体而言,它{ \em学习}来自真实数据的时间特征,并且{ \em指导}生成器预测未来帧。采用自适应滤波网络对视频中的空间一致性进行建模。为了进一步保证预测的时空一致性,还采用鉴别器来区分真实帧和生成帧。此外,运动控制器将信息泄漏给运动引导器和生成器以帮助运动的学习。所提出的LMVP可以有效地学习视频中的静态和时间特征,而无需人工标记。对合成数据和真实数据的实验表明,LMVP可以产生最先进的结果。

英文摘要:

We propose a Leaked Motion Video Predictor (LMVP) to predict future frames by capturing the spatial and temporal dependencies from given inputs. The motion is modeled by a newly proposed component, motion guider, which plays the role of both learner and teacher. Specifically, it {\em learns} the temporal features from real data and {\em guides} the generator to predict future frames. The spatial consistency in video is modeled by an adaptive filtering network. To further ensure the spatio-temporal consistency of the prediction, a discriminator is also adopted to distinguish the real and generated frames. Further, the discriminator leaks information to the motion guider and the generator to help the learning of motion. The proposed LMVP can effectively learn the static and temporal features in videos without the need for human labeling. Experiments on synthetic and real data demonstrate that LMVP can yield state-of-the-art results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑