Long-term time series forecasting (LTSF) has been widely applied in finance, traffic prediction, and other domains. Recently, patch-based transformers have emerged as a promising approach, segmenting data into sub-level patches that serve as input tokens. However, existing methods mostly rely on predetermined patch lengths, necessitating expert knowledge and posing challenges in capturing diverse characteristics across various scales. Moreover, time series data exhibit diverse variations and fluctuations across different temporal scales, which traditional approaches struggle to model effectively. In this paper, we propose a dynamic tokenizer with a dynamic sparse learning algorithm to capture diverse receptive fields and sparse patterns of time series data. In order to build hierarchical receptive fields, we develop a multi-scale Transformer model, coupled with multi-scale sequence extraction, capable of capturing multi-resolution features. Additionally, we introduce a group-aware rotary position encoding technique to enhance intra- and inter-group position awareness among representations across different temporal scales. Our proposed model, named DRFormer, is evaluated on various real-world datasets, and experimental results demonstrate its superiority compared to existing methods. Our code is available at: https://github.com/ruixindingECNU/DRFormer.
Cross-LKTCN: Modern Convolution Utilizing Cross-Variable Dependency for Multivariate Time Series Forecasting Dependency for Multivariate Time Series Forecasting
Performance-Carbon Trade-Offs across Architectural Biases in Shear Flow Forecasting
Department of Computer Science, University of Copenhagen, Denmark(丹麦哥本哈根大学计算机科学系);Niels Bohr Institute, University of Copenhagen, Denmark(丹麦哥本哈根大学尼尔斯·玻尔研究所)
Adaptive Convolutional Forecasting Network Based on Time Series Feature-Driven
School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);College of Design and Engineering, National University of Singapore(新加坡国立大学设计与工程学院)
Scale-Equivariant Generative Forecasting: Weight-Tied Dilated Convolutions, Wavelet Scattering Inputs, and Spectral-Consistency Training for Self-Similar Time Series
Attention Mamba: Time Series Modeling with Adaptive Pooling Acceleration and Receptive Field Enhancements
Graduate School of Information Science and Electrical Engineering, Kyushu University(九州大学情报科学与电气工程研究院);Graduate School of Business, ECUST, East China University of Science and Technology(华东理工大学商学院)
Improving Time Series Encoding with Noise-Aware Self-Supervised Learning and an Efficient Encoder
VinUni-Illinois Smart Health Center(VinUni-伊利诺伊智能健康中心);Cornell University(康奈尔大学);VinUniversity(VinUniversity(越南 VinUniversity 大学));Hanoi University of Science and Technology(河内理工大学);IBM Research, Thomas J. Watson Research Center(IBM 研究托马斯·J·沃森研究中心)