Towards Lightweight Time Series Forecasting: a Patch-wise Transformer with Weak Data Enriching
School of Data Science and Engineering, East China Normal University(华东师范大学数据科学与工程学院);School of Computer Science and Technology, Xidian University(西安电子科技大学计算机科学与技术学院);Shanghai Yunxi Technology(上海云玺科技)
Meng Wang, Jintao Yang, Bin Yang, Hui Li, Tongxin Gong, Bo Yang, Jiangtao Cui
备注:
Accepted by the 41st IEEE International Conference on Data Engineering (ICDE 2025)
Patch-wise Transformer based time series forecasting achieves superior accuracy. However, this superiority relies heavily on intricate model design with massive parameters, rendering both training and inference expensive, thus preventing their deployments on edge devices with limited resources and low latency requirements. In addition, existing methods often work in an autoregressive manner, which take into account only historical values, but ignore valuable, easy-to-obtain context information, such as weather forecasts, date and time of day. To contend with the two limitations, we propose LiPFormer, a novel Lightweight Patch-wise Transformer with weak data enriching. First, to simplify the Transformer backbone, LiPFormer employs a novel lightweight cross-patch attention and a linear transformation-based attention to eliminate Layer Normalization and Feed Forward Network, two heavy components in existing Transformers. Second, we propose a lightweight, weak data enriching module to provide additional, valuable weak supervision to the training. It enhances forecasting accuracy without significantly increasing model complexity as it does not involve expensive, human-labeling but using easily accessible context information. This facilitates the weak data enriching to plug-and-play on existing models. Extensive experiments on nine benchmark time series datasets demonstrate that LiPFormer outperforms state-of-the-art methods in accuracy, while significantly reducing parameter scale, training duration, and GPU memory usage. Deployment on an edge device reveals that LiPFormer takes only 1/3 inference time compared to classic Transformers. In addition, we demonstrate that the weak data enriching can integrate seamlessly into various Transformer based models to enhance their accuracy, suggesting its generality.
Advancing Long-Term Multi-Energy Load Forecasting with Patchformer: A Patch and Transformer-Based Approach
School of Mathematics, Statistics and Actuarial Science(数学、统计与精算科学学院);University of Essex(埃塞克斯大学);Alliance Manchester Business School(曼彻斯特联盟商学院);University of Manchester(曼彻斯特大学)
Structured Latent Space Modeling over Multi-Scale Temporal Patches for Multivariate Time Series Forecasting
School of Computing and Data Science, The University of Hong Kong(香港大学计算与数据科学学院);Innovation Engineering College, Macau University of Science and Technology(澳门科技大学创新工程学院)
PMDformer: Patch-Mean Decoupling Information Transformer for Long-term Forecasting
Southwestern University of Finance and Economics(西南财经大学);Shanghai Academy of AI for Science(上海人工智能研究院);Fudan University(复旦大学);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));X-Humanoid Research Institute(X-Humanoid 研究院);Chengdu Everimaging Science and Technology Co., Ltd.(成都永新科技有限公司);Artificial Intelligence and Digital Finance Key Laboratory of Sichuan Province(四川省人工智能与数字金融重点实验室)
PhaseFormer: From Patches to Phases for Efficient and Effective Time Series Forecasting
School of Computer Science and Engineering(计算机科学与工程学院);Beihang University(北航);Department of Computer Science and Engineering(计算机科学与工程系);The Hong Kong University of Science and Technology(香港科学与技术大学)
CT-PatchTST: Channel-Time Patch Time-Series Transformer for Long-Term Renewable Energy Forecasting
School of Electrical and Computer Engineering(电气与计算机工程学院);Cornell University(康奈尔大学);Department of Electrical and Computer Engineering(电气与计算机工程系);Northeastern University(东北大学);Fu Foundation School of Engineering and Applied Science(富兰克林基金会工程与应用科学学院);Columbia University in the City of New York(纽约市哥伦比亚大学);School of Engineering(工程学院);Santa Clara University(圣克拉拉大学);Department of Mechanical and Aerospace Engineering(机械与航空航天工程系);University of Houston(休斯顿大学)
Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift
College of Intelligence and Computing, Tianjin University, China(天津大学智能计算学院);I2R, Agency for Science, Technology and Research, Singapore(新加坡科技研究局I2R);Department of Computer Science, Khalifa University, UAE(阿联酋卡利法大学计算机科学系)
Airport Passenger Flow Forecasting via Deformable Temporal-Spectral Transformer Approach
School of Electronic and Information Engineering and State Key Laboratory of CNS/ATM, Beihang University(电子信息工程学院和 CNS/ATM 国家重点实验室,北京航空航天大学);Department of Information and Technology, Beijing Capital International Airport Co., Ltd.(信息与技术部,北京首都国际机场有限公司);College of Information and Electrical Engineering, China Agricultural University(信息与电气工程学院,中国农业大学);State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,中国科学院自动化研究所);School of Artificial Intelligence, University of the Chinese Academy of Sciences(人工智能学院,中国科学院大学)
Adaptive Patching Is Harder Than It Looks For Time-Series Forecasting
ICube, University of Strasbourg, Illkirch-Graffenstaden, France(斯特拉斯堡大学ICube研究所,法国伊尔克里奇-格拉夫芬斯坦德);Technical University of Munich(慕尼黑技术大学);FinTech Thrust, The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州)金融科技研究组);Computer Science Department, Hainan Bielefeld University of Applied Sciences(海南比尔费尔德应用科学大学计算机科学系);Cephalgo, Strasbourg, France(法国斯特拉斯堡Cephalgo公司);Heilbronn Data Science Center, Munich Data Science Institute(慕尼黑数据科学研究所海德堡数据科学中心)
TimeSQL: Improving Multivariate Time Series Forecasting with Multi-Scale Patching and Smooth Quadratic Loss
College of Electrical Engineering, Sichuan University(四川大学电气工程学院);College of Architecture and Environment, Sichuan University(四川大学建筑与环境学院);State Grid Sichuan Electric Power Research Institute(国网四川省电力公司电力科学研究院);College of Computer Science, Sichuan University(四川大学计算机学院)
SACQ: Structured Decoding with Memory-Conditioned Refinement for Long-Horizon Forecasting
University of Science and Technology Beijing(北京科技大学);Zhejiang University of Technology(浙江工业大学);Institution of Artificial Intelligence, University of Science and Technology Beijing(北京科技大学人工智能研究院)
MTSMAE: Masked Autoencoders for Multivariate Time-Series Forecasting
University of Science and Technology of China(中国科学技术大学);Institute of Advanced Technology, University of Science and Technology of China(中国科学技术大学先进技术研究院);Jiaxing University(嘉兴学院)
QKCV Attention: Enhancing Time Series Forecasting with Static Categorical Embeddings for Both Lightweight and Pre-trained Foundation Models
Independent Researcher(独立研究者);Key Laboratory of Brain-Machine Intelligence for Information Behavior (Ministry of Education and Shanghai)(信息行为脑机智能关键实验室);School of Business and Management(商学院)
Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
Monash University(莫纳什大学);Ant Group(蚂蚁集团);IBM Research(IBM研究院);Griffith University(格里菲斯大学);Alibaba Group(阿里巴巴集团);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))