Time series analysis is vital for numerous applications, and transformers have become increasingly prominent in this domain. Leading methods customize the transformer architecture from NLP and CV, utilizing a patching technique to convert continuous signals into segments. Yet, time series data are uniquely challenging due to significant distribution shifts and intrinsic noise levels. To address these two challenges,we introduce the Sparse Vector Quantized FFN-Free Transformer (Sparse-VQ). Our methodology capitalizes on a sparse vector quantization technique coupled with Reverse Instance Normalization (RevIN) to reduce noise impact and capture sufficient statistics for forecasting, serving as an alternative to the Feed-Forward layer (FFN) in the transformer architecture. Our FFN-free approach trims the parameter count, enhancing computational efficiency and reducing overfitting. Through evaluations across ten benchmark datasets, including the newly introduced CAISO dataset, Sparse-VQ surpasses leading models with a 7.84% and 4.17% decrease in MAE for univariate and multivariate time series forecasting, respectively. Moreover, it can be seamlessly integrated with existing transformer-based models to elevate their performance.
QUALS: Corpus Equilibrium for Universal Forecasting via Pattern Quantization and Learnability Synchronization
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所人工智能安全国家重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；School of Data Science and Engineering, East China Normal University（华东师范大学数据科学与工程学院）
Groundwater is a precious natural resource. Groundwater level (GWL) forecasting is crucial in the field of water resource management. Measurement of GWL from observation-wells is the principle source of information about the aquifer and is critical to its evaluation. Most part of the Udupi district of Karnataka State in India consists of geological formations: lateritic terrain and gneissic complex. Due to the topographical ruggedness and inconsistency in rainfall, the GWL in Udupi region is declining continually and most of the open wells are drying-up during the summer. Hence, the current re…
Indian Institute of Technology Kanpur（印度理工学院坎普尔分校）
The energy output a photo voltaic(PV) panel is a function of solar irradiation and weather parameters like temperature and wind speed etc. A general measure for solar irradiation called Global Horizontal Irradiance (GHI), customarily reported in Watt/meter$^2$, is a generic indicator for this intermittent energy resource. An accurate prediction of GHI is necessary for reliable grid integration of the renewable as well as for power market trading. While some machine learning techniques are well introduced along with the traditional time-series forecasting techniques, deep-learning techniques re…
Apollo-Forecast: Overcoming Aliasing and Inference Speed Challenges in Language Models for Time Series Forecasting
Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University（同济大学上海智能自主系统研究院）；College of Electronics and Information Engineering, Tongji University（同济大学电子与信息工程学院）；Ant Group（蚂蚁集团）；Huazhong University of Science and Technology（华中科技大学）；Macau University of Science and Technology（澳门科技大学）
Time series forecasting is a significant problem in many applications, e.g., financial predictions and business optimization. Modern datasets can have multiple correlated time series, which are often generated with global (shared) regularities and local (specific) dynamics. In this paper, we seek to tackle such forecasting problems with DeepDGL, a deep forecasting model that disentangles dynamics into global and local temporal patterns. DeepDGL employs an encoder-decoder architecture, consisting of two encoders to learn global and local temporal patterns, respectively, and a decoder to make mu…
Smart meter measurements, though critical for accurate demand forecasting, face several drawbacks including consumers' privacy, data breach issues, to name a few. Recent literature has explored Federated Learning (FL) as a promising privacy-preserving machine learning alternative which enables collaborative learning of a model without exposing private raw data for short term load forecasting. Despite its virtue, standard FL is still vulnerable to an intractable cyber threat known as Byzantine attack carried out by faulty and/or malicious clients. Therefore, to improve the robustness of federat…
ViTime: Foundation Model for Time Series Forecasting Powered by Vision Intelligence
Technion – Israel Institute of Technology（以色列理工学院）；Manchester Metropolitan University（曼彻斯特城市大学）；Northwest Polytechnical University（西北工业大学）；City University of Hong Kong（香港城市大学）
Rice University（莱斯大学）；Pennsylvania State University（宾夕法尼亚州立大学）；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）；University of Nevada, Reno（内华达大学雷诺分校）；Sapienza Universitá di Roma（罗马一大（萨皮恩扎大学））
Rocks under stress deform by creep mechanisms that include formation and slip on small-scale internal cracks. Intragranular cracks and slip along grain contacts release energy as elastic waves termed acoustic emissions (AE). AEs are thought to contain predictive information that can be used for fault failure forecasting. Here we present a method using unsupervised classification and an attention network to forecast labquakes using AE waveform features. Our data were generated in a laboratory setting using a biaxial shearing device with granular fault gouge intended to mimic the conditions of t…
AWS AI Labs（AWS人工智能实验室）；UC San Diego（加州大学圣迭戈分校）；University of Freiburg（弗莱堡大学）；Rutgers University（罗格斯大学）；Amazon Supply Chain Optimization Technologies（亚马逊供应链优化技术部）；UC Berkeley（加州大学伯克利分校）；New York University（纽约大学）
