arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1803.09760cs.CVcs.AIcs.LGcs.NE
收藏

用转型国家预测未来

Predicting the Future with Transformational States

University of Pennsylvania(宾夕法尼亚大学);Ryerson University(瑞尔森大学)

Andrew Jaegle, Oleh Rybkin, Konstantinos G. Derpanis, Kostas Daniilidis

摘要:

一个聪明的观察者观察世界,不仅看到什么,而且看到什么在移动,什么可以移动。换句话说,观察者看到了世界的现状在未来会如何变化。我们提出了一个模型,预测未来的图像,通过学习代表目前的状态和它的转换只给出了一个序列的图像。为此,我们引入了一个体系结构,该体系结构具有由两个组件组成的潜在状态,这两个组件分别用于捕获(i)当前图像状态和(ii)当前和未来状态之间的转换。我们将这种潜在状态与递归神经网络(RNN)核心耦合,该核心通过使用学习算子应用累积状态转换将过去状态转换为未来状态来预测未来帧。我们描述了如何将该模型集成到编码器-解码器卷积神经网络(CNN)架构中,该架构使用加权残差连接来整合过去的表示与未来的表示。从质量上讲,我们的方法生成的图像序列是稳定的,并在多个预测帧上捕捉逼真的运动,而不需要对抗性训练。从定量上看,我们的方法实现的预测结果与标准图像预测基准(Moving MNIST、KTH和UCF 101)的最新结果相当。

英文摘要:

An intelligent observer looks at the world and sees not only what is, but what is moving and what can be moved. In other words, the observer sees how the present state of the world can transform in the future. We propose a model that predicts future images by learning to represent the present state and its transformation given only a sequence of images. To do so, we introduce an architecture with a latent state composed of two components designed to capture (i) the present image state and (ii) the transformation between present and future states, respectively. We couple this latent state with a recurrent neural network (RNN) core that predicts future frames by transforming past states into future states by applying the accumulated state transformation with a learned operator. We describe how this model can be integrated into an encoder-decoder convolutional neural network (CNN) architecture that uses weighted residual connections to integrate representations of the past with representations of the future. Qualitatively, our approach generates image sequences that are stable and capture realistic motion over multiple predicted frames, without requiring adversarial training. Quantitatively, our method achieves prediction results comparable to state-of-the-art results on standard image prediction benchmarks (Moving MNIST, KTH, and UCF101).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑