Although the Transformer has been the dominant architecture for time series forecasting tasks in recent years, a fundamental challenge remains: the permutation-invariant self-attention mechanism within Transformers leads to a loss of temporal information. To tackle these challenges, we propose PatchMixer, a novel CNN-based model. It introduces a permutation-variant convolutional structure to preserve temporal information. Diverging from conventional CNNs in this field, which often employ multiple scales or numerous branches, our method relies exclusively on depthwise separable convolutions. This allows us to extract both local features and global correlations using a single-scale architecture. Furthermore, we employ dual forecasting heads encompassing linear and nonlinear components to better model future curve trends and details. Our experimental results on seven time-series forecasting benchmarks indicate that compared with the state-of-the-art method and the best-performing CNN, PatchMixer yields $3.9\%$ and $21.2\%$ relative improvements, respectively, while being 2-3x faster than the most advanced method.
CT-PatchTST: Channel-Time Patch Time-Series Transformer for Long-Term Renewable Energy Forecasting
School of Electrical and Computer Engineering(电气与计算机工程学院);Cornell University(康奈尔大学);Department of Electrical and Computer Engineering(电气与计算机工程系);Northeastern University(东北大学);Fu Foundation School of Engineering and Applied Science(富兰克林基金会工程与应用科学学院);Columbia University in the City of New York(纽约市哥伦比亚大学);School of Engineering(工程学院);Santa Clara University(圣克拉拉大学);Department of Mechanical and Aerospace Engineering(机械与航空航天工程系);University of Houston(休斯顿大学)
Advancing Long-Term Multi-Energy Load Forecasting with Patchformer: A Patch and Transformer-Based Approach
School of Mathematics, Statistics and Actuarial Science(数学、统计与精算科学学院);University of Essex(埃塞克斯大学);Alliance Manchester Business School(曼彻斯特联盟商学院);University of Manchester(曼彻斯特大学)
TimeSQL: Improving Multivariate Time Series Forecasting with Multi-Scale Patching and Smooth Quadratic Loss
College of Electrical Engineering, Sichuan University(四川大学电气工程学院);College of Architecture and Environment, Sichuan University(四川大学建筑与环境学院);State Grid Sichuan Electric Power Research Institute(国网四川省电力公司电力科学研究院);College of Computer Science, Sichuan University(四川大学计算机学院)
SACQ: Structured Decoding with Memory-Conditioned Refinement for Long-Horizon Forecasting
University of Science and Technology Beijing(北京科技大学);Zhejiang University of Technology(浙江工业大学);Institution of Artificial Intelligence, University of Science and Technology Beijing(北京科技大学人工智能研究院)
Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift
College of Intelligence and Computing, Tianjin University, China(天津大学智能计算学院);I2R, Agency for Science, Technology and Research, Singapore(新加坡科技研究局I2R);Department of Computer Science, Khalifa University, UAE(阿联酋卡利法大学计算机科学系)
Structured Latent Space Modeling over Multi-Scale Temporal Patches for Multivariate Time Series Forecasting
School of Computing and Data Science, The University of Hong Kong(香港大学计算与数据科学学院);Innovation Engineering College, Macau University of Science and Technology(澳门科技大学创新工程学院)
PMDformer: Patch-Mean Decoupling Information Transformer for Long-term Forecasting
Southwestern University of Finance and Economics(西南财经大学);Shanghai Academy of AI for Science(上海人工智能研究院);Fudan University(复旦大学);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));X-Humanoid Research Institute(X-Humanoid 研究院);Chengdu Everimaging Science and Technology Co., Ltd.(成都永新科技有限公司);Artificial Intelligence and Digital Finance Key Laboratory of Sichuan Province(四川省人工智能与数字金融重点实验室)
PhaseFormer: From Patches to Phases for Efficient and Effective Time Series Forecasting
School of Computer Science and Engineering(计算机科学与工程学院);Beihang University(北航);Department of Computer Science and Engineering(计算机科学与工程系);The Hong Kong University of Science and Technology(香港科学与技术大学)
Sliding-Window Reordering with Overlap Averaging: A Simple Time-Domain Augmentation for Multivariate Forecasting
Data Science Group, University of Hildesheim(希尔德斯海姆大学数据科学组);Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim(希尔德斯海姆大学信息系统与机器学习实验室)
LoFT-LLM: Low-Frequency Time-Series Forecasting with Large Language Models
School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)(计算机科学与技术学院,哈尔滨工业大学(深圳));The Chinese University of Hong Kong(香港中文大学)
AdaMixT: Adaptive Weighted Mixture of Multi-Scale Expert Transformers for Time Series Forecasting
School of Computer Science, Peking University(北京大学计算机科学系);Center for Machine Learning Research, Peking University(北京大学机器学习研究中心);Tsinghua University(清华大学)
Towards Lightweight Time Series Forecasting: a Patch-wise Transformer with Weak Data Enriching
School of Data Science and Engineering, East China Normal University(华东师范大学数据科学与工程学院);School of Computer Science and Technology, Xidian University(西安电子科技大学计算机科学与技术学院);Shanghai Yunxi Technology(上海云玺科技)
现实世界的各种应用依赖于未来的信息来做出决策,因此需要高效和准确的长序列多变量时间序列预测。虽然最近的基于注意力的预测模型在捕捉长期依赖性方面表现出很强的能力,但它们仍然受到两个关键限制。首先,规范自注意力具有二次复杂度w.r.t.输入时间序列长度,从而效率不足。其次,不同变量的时间序列通常具有不同的时间动态,现有的研究未能捕获,因为它们使用相同的模型参数空间,例如,投影矩阵,为所有变量的时间序列,从而在准确性不足。为了确保高效率和准确性,我们提出了Triformer,一个三角形,变量特定的注意。(i)线性复杂度:我们引入了一个新的补丁注意线性复杂度。当堆叠多层补丁注意力时,提出了一种三角形结构,使得层的大小呈指数收缩,从而保持线性复杂度。(ii)变量特定参数:我们提出了一种轻量级的方法,以使不同的变量的时间序列的不同的模型参数集,以提高准确性,而不损害效率和内存使用。来自多个领域的四个数据集的强有力的经验证据证明了我们的设计选择,它表明Triformer优于最先进的方法w.r.t.准确性和效率。这是“Triformer:Triangular,Variable-Specific Attentions for Long Sequence Multivariate Time Series Forecasting”的扩展版本,将出现在IJCAI 2022 [Cirstea et al.,…
TimeFound: A Foundation Model for Time Series Forecasting
Baidu Research(百度研究院);University of Science and Technology of China(中国科学技术大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Adaptive Patching Is Harder Than It Looks For Time-Series Forecasting
ICube, University of Strasbourg, Illkirch-Graffenstaden, France(斯特拉斯堡大学ICube研究所,法国伊尔克里奇-格拉夫芬斯坦德);Technical University of Munich(慕尼黑技术大学);FinTech Thrust, The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州)金融科技研究组);Computer Science Department, Hainan Bielefeld University of Applied Sciences(海南比尔费尔德应用科学大学计算机科学系);Cephalgo, Strasbourg, France(法国斯特拉斯堡Cephalgo公司);Heilbronn Data Science Center, Munich Data Science Institute(慕尼黑数据科学研究所海德堡数据科学中心)
UniDiff: A Unified Diffusion Framework for Multimodal Time Series Forecasting
School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, China(人工智能学院、光学与电子学(iOPEN)、西北工业大学,西安710072,中国);Institute of Artificial Intelligence (TeleAI), China Telecom, China(人工智能研究所(TeleAI)、中国电信,中国)
Airport Passenger Flow Forecasting via Deformable Temporal-Spectral Transformer Approach
School of Electronic and Information Engineering and State Key Laboratory of CNS/ATM, Beihang University(电子信息工程学院和 CNS/ATM 国家重点实验室,北京航空航天大学);Department of Information and Technology, Beijing Capital International Airport Co., Ltd.(信息与技术部,北京首都国际机场有限公司);College of Information and Electrical Engineering, China Agricultural University(信息与电气工程学院,中国农业大学);State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,中国科学院自动化研究所);School of Artificial Intelligence, University of the Chinese Academy of Sciences(人工智能学院,中国科学院大学)