The Transformer is a highly successful deep learning model that has revolutionised the world of artificial neural networks, first in natural language processing and later in computer vision. This model is based on the attention mechanism and is able to capture complex semantic relationships between a variety of patterns present in the input data. Precisely because of these characteristics, the Transformer has recently been exploited for time series forecasting problems, assuming a natural adaptability to the domain of continuous numerical series. Despite the acclaimed results in the literature, some works have raised doubts about the robustness and effectiveness of this approach. In this paper, we further investigate the effectiveness of Transformer-based models applied to the domain of time series forecasting, demonstrate their limitations, and propose a set of alternative models that are better performing and significantly less complex. In particular, we empirically show how simplifying Transformer-based forecasting models almost always leads to an improvement, reaching state of the art performance. We also propose shallow models without the attention mechanism, which compete with the overall state of the art in long time series forecasting, and demonstrate their ability to accurately predict time series over extremely long windows. From a methodological perspective, we show how it is always necessary to use a simple baseline to verify the effectiveness of proposed models, and finally, we conclude the paper with a reflection on recent research paths and the opportunity to follow trends and hypes even where it may not be necessary.
FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective
Beijing Institute of Technology（北京理工大学）；Tongji University（同济大学）；University of Oxford（牛津大学）；University of Macau（澳门大学）；HeFei University of Technology（合肥工业大学）；Macquarie University（麦考瑞大学）
Rethinking Urban Mobility Prediction: A Super-Multivariate Time Series Forecasting Approach
Sichuan University（四川大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；McGill University（麦吉尔大学）；Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Lab（上海人工智能实验室）
CMoS: Rethinking Time Series Prediction Through the Lens of Chunk-wise Spatial Correlations
Computer Network Information Center, CAS（中国科学院计算机网络信息中心）；University of Chinese Academy of Sciences（中国科学院大学）；Nanjing University（南京大学）；Tsinghua University（清华大学）
Rethinking Recurrent Neural Networks for Time Series Forecasting: A Reinforced Recurrent Encoder with Prediction-Oriented Proximal Policy Optimization
School of Management, Huazhong University of Science and Technology（华中科技大学管理学院）；Center for Applied Mathematics, École des Ponts ParisTech（巴黎高等理工学院应用数学中心）；School of Mechanical Engineering, Dalian Jiaotong University（大连交通大学机械工程学院）；School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）
NeuTSFlow: Modeling Continuous Functions Behind Time Series Forecasting
University of Science and Technology of China（中国科学技术大学）；Tianjin University（天津大学）；Xi’an Jiaotong University（西安交通大学）；University of Cambridge（剑桥大学）；YMSC, Tsinghua University（清华大学数学科学中心）
Rethinking Graph Contrastive Learning through Relative Similarity Preservation
Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）；University of Chinese Academy of Sciences（中国科学院大学）；Hangzhou Institute for Advanced Study, UCAS（国科大杭州高等研究院）；School of Computing and Information Technology, Great Bay University（大湾区大学计算与信息技术学院）；Department of Computer and Information Science, IOTSC, University of Macau（澳门大学计算机与信息科学系）
Rethinking Dimensional Rationale in Graph Contrastive Learning from Causal Perspective
Science & Technology on Integrated Information System Laboratory, Institute of Software Chinese Academy of Sciences（中国科学院软件研究所综合信息系统技术国家级重点实验室）；State Key Laboratory of Intelligent Game（智能博弈国家重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；State Key Laboratory of Computer Science, Institute of Software Chinese Academy of Sciences（中国科学院软件研究所计算机科学国家重点实验室）
Curse of Attention: A Kernel-Based Perspective for Why Transformers Fail to Generalize on Time Series Forecasting and Beyond
The University of Hong Kong（香港大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；The Simons Institute for the Theory of Computing（西蒙斯理论计算研究所）；UC Berkeley（加州大学伯克利分校）；Sun Yat-sen University（中山大学）
Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting
Morgan Stanley（摩根士丹利）；ServiceNow Research（ServiceNow 研究院）；Université de Montréal（蒙特利尔大学）；Mila-Quebec AI Institute（米拉-魁北克人工智能研究所）；McGill University（麦吉尔大学）
