arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1412.6604cs.LGcs.CV
收藏

视频(语言)建模:自然视频生成模型的基线

Video (language) modeling: a baseline for generative models of natural videos

Facebook AI Research(脸书人工智能研究院);Courant Institute of Mathematical Sciences(库朗数学科学研究所)

MarcAurelio Ranzato, Arthur Szlam, Joan Bruna, Michael Mathieu, Ronan Collobert, Sumit Chopra

摘要:

我们提出了一个强大的基线模型,用于使用视频数据进行无监督特征学习。通过学习预测丢失的帧或从输入视频序列中推断未来的帧,该模型发现了空间和时间相关性,这对于表示复杂的变形和运动模式是有用的。我们提出的模型在很大程度上是从语言建模文献中借来的,并通过将图像块的空间量化到一个大的字典中来适应视觉领域。我们展示了填充和生成任务的方法。这是第一次,我们表明,在自然视频上训练后,这样的模型可以预测短视频序列上的非平凡运动。

英文摘要:

We propose a strong baseline model for unsupervised feature learning using video data. By learning to predict missing frames or extrapolate future frames from an input video sequence, the model discovers both spatial and temporal correlations which are useful to represent complex deformations and motion patterns. The models we propose are largely borrowed from the language modeling literature, and adapted to the vision domain by quantizing the space of image patches into a large dictionary. We demonstrate the approach on both a filling and a generation task. For the first time, we show that, after training on natural videos, such a model can predict non-trivial motions over short video sequences.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑