Transformer-based models have gained large popularity and demonstrated promising results in long-term time-series forecasting in recent years. In addition to learning attention in time domain, recent works also explore learning attention in frequency domains (e.g., Fourier domain, wavelet domain), given that seasonal patterns can be better captured in these domains. In this work, we seek to understand the relationships between attention models in different time and frequency domains. Theoretically, we show that attention models in different domains are equivalent under linear conditions (i.e., linear kernel to attention scores). Empirically, we analyze how attention models of different domains show different behaviors through various synthetic experiments with seasonality, trend and noise, with emphasis on the role of softmax operation therein. Both these theoretical and empirical analyses motivate us to propose a new method: TDformer (Trend Decomposition Transformer), that first applies seasonal-trend decomposition, and then additively combines an MLP which predicts the trend component with Fourier attention which predicts the seasonal component to obtain the final prediction. Extensive experiments on benchmark time-series forecasting datasets demonstrate that TDformer achieves state-of-the-art performance against existing attention-based models.
Forecasting the behavior of other agents is an integral part of the modern robotic autonomy stack, especially in safety-critical scenarios with human-robot interaction, such as autonomous driving. In turn, there has been a significant amount of interest and research in trajectory forecasting, resulting in a wide variety of approaches. Common to all works, however, is the use of the same few accuracy-based evaluation metrics, e.g., displacement error and log-likelihood. While these metrics are informative, they are task-agnostic and predictions that are evaluated as equal can lead to vastly dif…
FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective
Beijing Institute of Technology（北京理工大学）；Tongji University（同济大学）；University of Oxford（牛津大学）；University of Macau（澳门大学）；HeFei University of Technology（合肥工业大学）；Macquarie University（麦考瑞大学）
CMoS: Rethinking Time Series Prediction Through the Lens of Chunk-wise Spatial Correlations
Computer Network Information Center, CAS（中国科学院计算机网络信息中心）；University of Chinese Academy of Sciences（中国科学院大学）；Nanjing University（南京大学）；Tsinghua University（清华大学）
Rethinking Recurrent Neural Networks for Time Series Forecasting: A Reinforced Recurrent Encoder with Prediction-Oriented Proximal Policy Optimization
School of Management, Huazhong University of Science and Technology（华中科技大学管理学院）；Center for Applied Mathematics, École des Ponts ParisTech（巴黎高等理工学院应用数学中心）；School of Mechanical Engineering, Dalian Jiaotong University（大连交通大学机械工程学院）；School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）
Multi-horizon probabilistic time series forecasting has wide applicability to real-world tasks such as demand forecasting. Recent work in neural time-series forecasting mainly focus on the use of Seq2Seq architectures. For example, MQTransformer - an improvement of MQCNN - has shown the state-of-the-art performance in probabilistic demand forecasting. In this paper, we consider incorporating cross-entity information to enhance model performance by adding a cross-entity attention mechanism along with a retrieval mechanism to select which entities to attend over. We demonstrate how our new neura…
Rethinking Urban Mobility Prediction: A Super-Multivariate Time Series Forecasting Approach
Sichuan University（四川大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；McGill University（麦吉尔大学）；Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Lab（上海人工智能实验室）
MRC Centre for Global Infectious Disease Analysis, Imperial College London（帝国理工学院MRC全球传染病分析中心）；Imperial College Healthcare NHS Trust（帝国理工学院医疗保健国民健康服务体系信托）；Faculty of Medicine, Imperial College London（帝国理工学院医学院）；Department of Infectious Disease, Imperial College London（帝国理工学院传染病系）；Department of Mathematics, Imperial College London（帝国理工学院数学系）
There were 25.6 million attendances at Emergency Departments (EDs) in England in 2019 corresponding to an increase of 12 million attendances over the past ten years. The steadily rising demand at EDs creates a constant challenge to provide adequate quality of care while maintaining standards and productivity. Managing hospital demand effectively requires an adequate knowledge of the future rate of admission. Using 8 years of electronic admissions data from two major acute care hospitals in London, we develop a novel ensemble methodology that combines the outcomes of the best performing time se…
In this work, we propose a deep learning approach to forecasting state-level COVID-19 trends of weekly cumulative death in the United States (US) and incident cases in Germany. This approach includes a transformer model, an ensemble method, and a data augmentation technique for time series. We arrange the inputs of the transformer in such a way that predictions for different states can attend to the trends of the others. To overcome the issue of scarcity of training data for this COVID-19 pandemic, we have developed a novel data augmentation technique to generate useful data for training. More…
The problem of (point) forecasting $ \textit{univariate} $ time series is considered. Most approaches, ranging from traditional statistical methods to recent learning-based techniques with neural networks, directly operate on raw time series observations. As an extension, we study whether $\textit{local topological properties}$, as captured via persistent homology, can serve as a reliable signal that provides complementary information for learning to forecast. To this end, we propose $\textit{topological attention}$, which allows attending to local topological features within a time horizon of…
We propose an efficient design of Transformer-based models for multivariate time series forecasting and self-supervised representation learning. It is based on two key components: (i) segmentation of time series into subseries-level patches which are served as input tokens to Transformer; (ii) channel-independence where each channel contains a single univariate time series that shares the same embedding and Transformer weights across all the series. Patching design naturally has three-fold benefit: local semantic information is retained in the embedding; computation and memory usage of the att…
FMamba: Mamba based on Fast-attention for Multivariate Time-series Forecasting
Institute of Advanced Technology, USTC（中国科学技术大学先进技术研究院）；Department of Automation, USTC（中国科学技术大学自动化系）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
More Sensors Only One Field: Rethinking Continual Spatio-Temporal Forecasting
City University of Hong Kong (Dongguan)（香港城市大学（东莞））；City University of Hong Kong（香港城市大学）；Zhuhai College of Science and Technology（珠海科技学院）；Northwestern Polytechnical University（西北工业大学）
Smart Metro: Deep Learning Approaches to Forecasting the MRT Line 3 Ridership
Technological University of the Philippines（菲律宾技术大学）；National University（国立大学）；Philippine Coding Camp（菲律宾编程营）；University of the Philippines Open University（菲律宾大学开放大学）
