从纯合成数据中学习时空基础模型
Learning Spatio-Temporal Foundation Models from Pure Synthetic Data
浏览论文内容
中文总结 AI 辅助
研究旨在学习时空基础模型，提出NeoST，通过在程序生成的合成系统上预训练，引入可扩展语料库、潜在空间推理架构和目标，实验证明其在多样真实世界时空系统中性能优越，有长期稳定性和推理效率。
中文摘要 AI 辅助
时空基础模型（STFMs）旨在学习复杂动力系统在时空上的可泛化表示。现有方法存在诸多问题，如真实世界预训练数据的分布偏差、自回归或基于扩散范式的结构瓶颈以及过度强调噪声观测中点状重建的目标。本文提出了NeoST，首个仅在程序生成的合成系统上预训练的时空基础模型。它引入可扩展合成预训练语料库减轻真实世界偏差，有潜在空间推理架构及潜在空间目标。实验表明NeoST在多样真实世界时空系统中优于现有模型，具有卓越的长期稳定性和推理效率。
英文摘要
Spatio-Temporal Foundation Models (STFMs) aim to learn generalizable representations of complex dynamical systems across space and time. However, existing approaches suffer from distributional bias in real-world pre-training data, structural bottlenecks of autoregressive or diffusion-based paradigms, and objectives that overemphasize point-wise reconstruction in noisy observation space.We propose \textbf{NeoST}, the first spatio-temporal foundation model pre-trained solely on procedurally generated synthetic systems. NeoST introduces a scalable synthetic pre-training corpus to mitigate real-world bias, a latent-space reasoning architecture that generates and iteratively refines multiple future trajectories without sequential error accumulation, and latent-space objectives that emphasize structural dynamics and enable inference-time correction under distribution shifts.Extensive experiments across diverse real-world benchmarks show that NeoST consistently outperforms existing STFMs in diverse real-world spatio-temporal systems, achieves superior long-horizon stability and inference efficiency.
发表机构
- Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
- National University of Singapore（新加坡国立大学）
- Hong Kong University of Science and Technology（香港科技大学）
- Hong Kong Polytechnic University（香港理工大学）
机构由 AI 辅助整理，请以论文原文为准。
