arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.09723cs.LG
收藏

sTransformer:一种用于时间序列预测的提取序列间与时间信息的模块化方法

sTransformer: A Modular Approach for Extracting Inter-Sequential and Temporal Information for Time-Series Forecasting

Tsinghua University(清华大学)

Jiaheng Yin, Zhengxin Shi, Jianshen Zhang, Xiaomin Lin, Yulin Huang, Yongzhi Qi, Wei Qi

AI总结:

针对现有Transformer类时间序列预测模型的两类局限,提出融合STCN与序列引导掩码注意力的模块化sTransformer,在长期预测等任务上取得SOTA,验证了Transformer结构的有效性。

摘要:

近年来,众多基于Transformer的模型被应用于长期时间序列预测(LTSF)任务。然而,近期针对线性模型的研究对其有效性提出了质疑,表明简单的线性层性能可优于复杂的基于Transformer的模型。本研究回顾了现有基于Transformer的模型并将其分为两大类:(1)模型结构修改类;(2)输入数据修改类。前者具备可扩展性,但在捕捉序列间信息方面存在不足;后者对时间序列数据进行预处理,但难以作为可扩展模块使用。我们提出sTransformer,该模型引入序列与时间卷积网络(Sequence and Temporal Convolutional Network,STCN)以充分捕捉序列与时间信息。此外,我们引入序列引导掩码注意力机制以捕捉全局特征信息。我们的方法在保证模块可扩展性的同时,确保能够捕捉序列间信息。我们在长期时间序列预测任务上将该模型与线性模型及现有预测模型进行对比,取得了全新的最优(state-of-the-art)结果。我们还在其他时间序列任务上开展了实验,均取得了优异性能。这些结果表明,基于Transformer的结构依然有效,且我们的模型可作为时间序列任务的可靠基准。

英文摘要:

In recent years, numerous Transformer-based models have been applied to long-term time-series forecasting (LTSF) tasks. However, recent studies with linear models have questioned their effectiveness, demonstrating that simple linear layers can outperform sophisticated Transformer-based models. In this work, we review and categorize existing Transformer-based models into two main types: (1) modifications to the model structure and (2) modifications to the input data. The former offers scalability but falls short in capturing inter-sequential information, while the latter preprocesses time-series data but is challenging to use as a scalable module. We propose $\textbf{sTransformer}$, which introduces the Sequence and Temporal Convolutional Network (STCN) to fully capture both sequential and temporal information. Additionally, we introduce a Sequence-guided Mask Attention mechanism to capture global feature information. Our approach ensures the capture of inter-sequential information while maintaining module scalability. We compare our model with linear models and existing forecasting models on long-term time-series forecasting, achieving new state-of-the-art results. We also conducted experiments on other time-series tasks, achieving strong performance. These demonstrate that Transformer-based structures remain effective and our model can serve as a viable baseline for time-series tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑