S-HR-VQVAE：用于视频预测的序列化层次残差学习向量量化变分自编码器
S-HR-VQVAE: Sequential Hierarchical Residual Learning Vector Quantized Variational Autoencoder for Video Prediction
AI总结：
本文提出S-HR-VQVAE视频预测模型，结合层次残差向量量化变分自编码器与自回归时空预测模型，在四个基准上以更小模型规模取得优于最先进方法的性能。
摘要：
我们通过提出一种新颖的模型来解决视频预测任务，该模型结合了（i）一种新颖的层次残差学习向量量化变分自编码器（HR-VQVAE），以及（ii）一种新颖的自回归时空预测模型（AST-PM）。我们将这种方法称为序列化层次残差学习向量量化变分自编码器（S-HR-VQVAE）。通过利用HR-VQVAE在利用简洁表示对静态图像建模方面的内在能力，并结合AST-PM处理时空信息的能力，S-HR-VQVAE能够更好地应对视频预测中的主要挑战。这些挑战包括学习时空信息、处理高维数据、对抗模糊预测以及隐式建模物理特性。在四个具有挑战性的任务上的大量实验结果表明，即KTH Human Action、TrafficBJ、Human3.6M和Kitti，我们的模型在定量和定性评估中均优于最先进的视频预测技术，尽管模型规模小得多。最后，我们通过提出一种新颖的训练方法来联合估计HR-VQVAE和AST-PM的参数，从而提升了S-HR-VQVAE的性能。
英文摘要：
We address the video prediction task by putting forth a novel model that combines (i) a novel hierarchical residual learning vector quantized variational autoencoder (HR-VQVAE), and (ii) a novel autoregressive spatiotemporal predictive model (AST-PM). We refer to this approach as a sequential hierarchical residual learning vector quantized variational autoencoder (S-HR-VQVAE). By leveraging the intrinsic capabilities of HR-VQVAE at modeling still images with a parsimonious representation, combined with the AST-PM's ability to handle spatiotemporal information, S-HR-VQVAE can better deal with major challenges in video prediction. These include learning spatiotemporal information, handling high dimensional data, combating blurry prediction, and implicit modeling of physical characteristics. Extensive experimental results on four challenging tasks, namely KTH Human Action, TrafficBJ, Human3.6M, and Kitti, demonstrate that our model compares favorably against state-of-the-art video prediction techniques both in quantitative and qualitative evaluations despite a much smaller model size. Finally, we boost S-HR-VQVAE by proposing a novel training method to jointly estimate the HR-VQVAE and AST-PM parameters.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
