arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.14537cs.LG
收藏

ST-ReP:高效学习用于时空预测的预测性表示

ST-ReP: Learning Predictive Representations Efficiently for Spatial-Temporal Forecasting

Tongji University(同济大学)

Qi Zheng, Zihao Yao, Yaying Zhang

AI总结:

提出轻量级表示学习模型ST-ReP,通过整合当前值重建与未来值预测的预训练框架、新型时空编码器及多时间尺度自监督损失,解决时空预测中的负样本对选择、空间相关性与可扩展性挑战,性能超越基线。

摘要:

时空预测在交通、能源和气候等多个领域至关重要且应用广泛。得益于大量无标签时空数据,自监督方法被越来越多地用于学习时空表示。然而,这面临三个关键挑战:1)由于变量的同质性,难以选择可靠的负样本对,阻碍了对比学习方法;2)忽视了变量随时间变化的空间相关性;3)现有自监督学习方法在效率和可扩展性方面存在局限。为解决这些问题,我们提出了一种轻量级的表示学习模型 ST-ReP,将当前值重建和未来值预测集成到用于时空预测的预训练框架中。我们设计了一个新的时空编码器来建模细粒度关系。此外,将多时间尺度分析纳入自监督损失以增强预测能力。跨多个领域的实验结果表明,所提出的模型超越了基于预训练的基线方法,展示了其学习紧凑且语义丰富的表示的能力,同时表现出卓越的可扩展性。

英文摘要:

Spatial-temporal forecasting is crucial and widely applicable in various domains such as traffic, energy, and climate. Benefiting from the abundance of unlabeled spatial-temporal data, self-supervised methods are increasingly adapted to learn spatial-temporal representations. However, it encounters three key challenges: 1) the difficulty in selecting reliable negative pairs due to the homogeneity of variables, hindering contrastive learning methods; 2) overlooking spatial correlations across variables over time; 3) limitations of efficiency and scalability in existing self-supervised learning methods. To tackle these, we propose a lightweight representation-learning model ST-ReP, integrating current value reconstruction and future value prediction into the pre-training framework for spatial-temporal forecasting. And we design a new spatial-temporal encoder to model fine-grained relationships. Moreover, multi-time scale analysis is incorporated into the self-supervised loss to enhance predictive capability. Experimental results across diverse domains demonstrate that the proposed model surpasses pre-training-based baselines, showcasing its ability to learn compact and semantically enriched representations while exhibiting superior scalability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑