动态物理模拟的视频预测：像素空间时空Transformer
Video Prediction of Dynamic Physical Simulations With Pixel-Space Spatiotemporal Transformers
AI总结：
本文提出一种纯Transformer像素空间自回归模型，用于动态物理模拟的视频预测，无需复杂训练或潜在特征学习，将物理准确预测时间范围延长50%，并验证了可解释性及对PDE参数估计的泛化能力。
摘要：
受自回归大语言模型（LLM）的性能和可扩展性启发，基于Transformer的模型近期在视觉领域取得了成功。本研究探讨了一种用于视频预测的Transformer适配方法，采用简单的端到端方法，比较了各种时空自注意力布局。我们聚焦于物理模拟随时间的因果建模——现有视频生成方法的一个常见缺陷，并尝试通过物理对象跟踪指标和在物理模拟数据集上的无监督训练来隔离时空推理。我们引入了一个简单而有效的纯Transformer模型用于自回归视频预测，利用连续像素空间表示进行视频预测。无需复杂的训练策略或潜在特征学习组件，我们的方法相比现有潜在空间方法，将物理上准确的预测时间范围显著延长了多达50%，同时在常见视频质量指标上保持相当的性能。此外，我们进行了可解释性实验，通过探测模型识别编码有用信息的网络区域，以准确估计PDE模拟参数，并发现这能泛化到分布外模拟参数的估计。这项工作为通过简单、参数高效且可解释的方法进一步进行基于注意力的视频时空建模提供了一个平台。
英文摘要：
Inspired by the performance and scalability of autoregressive large language models (LLMs), transformer-based models have seen recent success in the visual domain. This study investigates a transformer adaptation for video prediction with a simple end-to-end approach, comparing various spatiotemporal self-attention layouts. Focusing on causal modeling of physical simulations over time; a common shortcoming of existing video-generative approaches, we attempt to isolate spatiotemporal reasoning via physical object tracking metrics and unsupervised training on physical simulation datasets. We introduce a simple yet effective pure transformer model for autoregressive video prediction, utilizing continuous pixel-space representations for video prediction. Without the need for complex training strategies or latent feature-learning components, our approach significantly extends the time horizon for physically accurate predictions by up to 50% when compared with existing latent-space approaches, while maintaining comparable performance on common video quality metrics. In addition, we conduct interpretability experiments to identify network regions that encode information useful to perform accurate estimations of PDE simulation parameters via probing models, and find that this generalizes to the estimation of out-of-distribution simulation parameters. This work serves as a platform for further attention-based spatiotemporal modeling of videos via a simple, parameter efficient, and interpretable approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
