联合嵌入预测架构预训练是否有助于时间序列预测？
Does Joint-Embedding Predictive Architecture Pretraining Help Time Series Forecasting?
浏览论文内容
中文总结 AI 辅助
本研究大规模评估JEPA预训练在九种骨干网络和十一个基准上的效果，发现其收益因架构而异，既有增益也有退化，表明该变异性是普遍属性，选择骨干网络时需考虑。
中文摘要 AI 辅助
联合嵌入预测架构（JEPA）已成为时间序列领域一种有前景的自监督预训练范式，其通过在潜在空间中预测目标嵌入而非重建原始信号来学习表征。然而，关于其益处的证据仍然不一，且多数研究仅测试单一骨干网络或狭窄的架构集合，使得JEPA预训练是可靠的改进还是高度依赖于下游模型这一问题尚不明确。我们通过一项大规模评估来填补这一空白，该评估针对一个JEPA实例，跨越九个骨干网络和十一个基准，涵盖时间与时空预测，这是迄今为止对时间序列JEPA最广泛的跨架构评估。我们发现，该实例的收益在不同骨干网络间差异显著，对某些架构产生持续增益，而对另一些架构则造成持续退化，即使在同一数据集上也是如此。这一模式在两类任务中均成立，表明这种变异性是该实例的一般属性，而非数据集特定的伪影，在实际选择骨干网络时值得考虑。
英文摘要
Joint-embedding predictive architectures (JEPA) have emerged as a promising self-supervised pretraining paradigm for time series, learning representations by predicting target embeddings in latent space rather than reconstructing raw signals. Yet evidence on their benefits remains mixed, and most studies test only a single backbone or a narrow set of architectures, leaving unclear whether JEPA pretraining is a reliable improvement or one that depends heavily on the downstream model. We address this gap through a large scale evaluation of one JEPA instantiation across nine backbones and eleven benchmarks spanning temporal and spatio-temporal forecasting, the most extensive cross architecture assessment of JEPA for time series to date. We find that the benefit of this instantiation varies sharply across backbones, producing consistent gains for some architectures and consistent degradation for others, even on the same dataset. This pattern holds across both task families, indicating the variability is a general property of this instantiation rather than a dataset specific artifact worth accounting for when choosing a backbone in practice.
