近年来,众多基于Transformer的模型被应用于长期时间序列预测(LTSF)任务。然而,近期针对线性模型的研究对其有效性提出了质疑,表明简单的线性层性能可优于复杂的基于Transformer的模型。本研究回顾了现有基于Transformer的模型并将其分为两大类:(1)模型结构修改类;(2)输入数据修改类。前者具备可扩展性,但在捕捉序列间信息方面存在不足;后者对时间序列数据进行预处理,但难以作为可扩展模块使用。我们提出sTransformer,该模型引入序列与时间卷积网络(Sequence and Temporal Convolutional Network,STCN)以充分捕捉序列与时间信息。此外,我们引入序列引导掩码注意力机制以捕捉全局特征信息。我们的方法在保证模块可扩展性的同时,确保能够捕捉序列间信息。我们在长期时间序列预测任务上将该模型与线性模型及现有预测模型进行对比,取得了全新的最优(state-of-the-art)结果。我们还在其他时间序列任务上开展了实验,均取得了优异性能。这些结果表明,基于Transformer的结构依然有效,且我们的模型可作为时间序列任务的可靠基准。
英文摘要:
In recent years, numerous Transformer-based models have been applied to long-term time-series forecasting (LTSF) tasks. However, recent studies with linear models have questioned their effectiveness, demonstrating that simple linear layers can outperform sophisticated Transformer-based models. In this work, we review and categorize existing Transformer-based models into two main types: (1) modifications to the model structure and (2) modifications to the input data. The former offers scalability but falls short in capturing inter-sequential information, while the latter preprocesses time-series data but is challenging to use as a scalable module. We propose $\textbf{sTransformer}$, which introduces the Sequence and Temporal Convolutional Network (STCN) to fully capture both sequential and temporal information. Additionally, we introduce a Sequence-guided Mask Attention mechanism to capture global feature information. Our approach ensures the capture of inter-sequential information while maintaining module scalability. We compare our model with linear models and existing forecasting models on long-term time-series forecasting, achieving new state-of-the-art results. We also conducted experiments on other time-series tasks, achieving strong performance. These demonstrate that Transformer-based structures remain effective and our model can serve as a viable baseline for time-series tasks.
SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameters
School of Computer Science and Engineering, South China University of Technology(华南理工大学计算机科学与工程学院);Peng Cheng Laboratory(鹏城实验室);College of Information Science and Technology, Jinan University(暨南大学信息科学技术学院)
WaveCatBoost for Probabilistic Forecasting of Regional Air Quality Data
NIT Meghalaya(梅加拉亚国家理工学院);Sorbonne University(索邦大学);Sorbonne Center for Artificial Intelligence(索邦人工智能中心);Universiti Kebangsaan(马来西亚国民大学);University of Philippines(菲律宾大学);Institute of Environmental Science and Meteorology(环境科学与气象研究所)
Adaptive Convolutional Forecasting Network Based on Time Series Feature-Driven
School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);College of Design and Engineering, National University of Singapore(新加坡国立大学设计与工程学院)
Boosting MLPs with a Coarsening Strategy for Long-Term Time Series Forecasting
School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院);School of Biology & Basic Medical Science, Soochow University(苏州大学生物与基础医学院);School of Physics and Information Technology, Shaanxi Normal University(陕西师范大学物理与信息技术学院)
FMamba: Mamba based on Fast-attention for Multivariate Time-series Forecasting
Institute of Advanced Technology, USTC(中国科学技术大学先进技术研究院);Department of Automation, USTC(中国科学技术大学自动化系);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院)
WaveRoRA: Wavelet Rotary Route Attention for Multivariate Time Series Forecasting
School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications(北京邮电大学计算机学院(国家示范性软件学院));School of Computer Science and Engineering, University of Texas at Arlington(德克萨斯大学阿灵顿分校计算机科学与工程学院)
Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting
AMSS Center of Forecasting Science, Chinese Academy of Sciences(中国科学院预测科学中心);Department of Computer Science, Northwestern University(西北大学计算机科学系);Department of Statistics and Data Science, Northwestern University(西北大学统计与数据科学系);Department of Statistics and Operations Research, University of North Carolina(北卡罗来纳大学统计与运筹学系)
MCDFN: Supply Chain Demand Forecasting via an Explainable Multi-Channel Data Fusion Network Model
Khulna University of Engineering and Technology (KUET)(库尔纳工程技术大学);Okinawa Institute of Science and Technology Graduate University (OIST)(冲绳科学技术大学院大学);Hamad Bin Khalifa University(哈马德·本·哈利法大学)
Not All News Is Equal: Topic- and Event-Conditional Sentiment from Finetuned LLMs for Aluminum Price Forecasting
International Business School, Zhejiang University(浙江大学国际商学院);Center for Data Science, Zhejiang University(浙江大学数据科学中心);Centre for Human Genetics, Nuffield Department of Medicine, Oxford University(牛津大学纳菲尔德医学部人类遗传学中心);School of Medicine, Zhejiang University(浙江大学医学院);Bielefeld School of Business, Hochschule Bielefeld (HSBI) - University of Applied Sciences and Arts(比勒费尔德应用科学大学比勒费尔德商学院)
Double-Path Adaptive-correlation Spatial-Temporal Inverted Transformer for Stock Time Series Forecasting
School of Intelligence Science and Technology(智能科学与技术学院);Computational Intelligence Laboratory(计算智能实验室);National Key Laboratory of General Artificial Intelligence(通用人工智能全国重点实验室);Key Laboratory of Machine Perceptron (MOE)(教育部机器感知重点实验室)
LLM-PS: Empowering Large Language Models for Time Series Forecasting with Temporal Patterns and Semantics
Nanjing University of Science and Technology(南京理工大学);Nanjing University(南京大学);Shanghai Jiao Tong University(上海交通大学);Wuhan University(武汉大学);Nanyang Technological University(南洋理工大学)
VCformer: Variable Correlation Transformer with Inherent Lagged Correlation for Multivariate Time Series Forecasting
College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机与软件学院);National Engineering Laboratory for Big Data System Computing Technology, Shenzhen University(深圳大学国家大数据系统计算技术工程实验室)
Satellite-based feature extraction and multivariate time-series prediction of biotoxin contamination in shellfish
NOVA School of Science and Technology (NOVA FCT)(新里斯本大学科学与技术学院(NOVA FCT));IPMA - Instituto Português do Mar e da Atmosfera(葡萄牙海洋与大气研究所(IPMA));NOVA Laboratory for Computer Science and Informatics (NOVA LINCS)(新里斯本大学计算机科学与信息学实验室(NOVA LINCS));Center for Mathematics and Applications (NOVA Math)(新里斯本大学数学与应用中心(NOVA Math));NOVA Research & Development Unit for Mechanical and Industrial Engineering (UNIDEMI)(新里斯本大学机械与工业工程研发单元(UNIDEMI))
DeepVol: Volatility Forecasting from High-Frequency Data with Dilated Causal Convolutions
Oxford-Man Institute of Quantitative Finance, University of Oxford(牛津大学牛津-曼量化金融研究院);Universidad Carlos III de Madrid(马德里卡洛斯三世大学);University of Oxford(牛津大学)
Parallel BiLSTM-Transformer networks for forecasting chaotic dynamics
1 Department of Physics, Tianshui Normal University, Tianshui 741001, Gansu, China 2 Key Laboratory of Atomic;Molecular Physics \& Functional Material of Gansu Province, College of Physics;Electronic Engineering, Northwest Normal University, Lanzhou 730070, China 3 Lanzhou Center for Theoretical Physics, Lanzhou University, Lanzhou 730000, Gansu, China 4 Key Laboratory for Quantum Theory;Applications of the Ministry of Education, Lanzhou University, Lanzhou, Gansu 730000, China 5 Department of Physics, Xiamen University, Xiamen 361005, Fujian, China
Epidemiology-informed Graph Neural Network for Heterogeneity-aware Epidemic Forecasting
University of Canberra(堪培拉大学);University of Queensland(昆士兰大学);Hong Kong Polytechnic University(香港理工大学);Griffith University(格里菲斯大学);South China Normal University(华南师范大学)