In multivariate time series (MTS) forecasting, many deep learning based methods have been proposed for modeling dependencies at multiple spatial (inter-variate) or temporal (intra-variate) scales. However, existing methods may fail to model dependencies across multiple spatial-temporal scales (ST-scales, i.e., scales that jointly consider spatial and temporal scopes). In this work, we propose ST-Hyper to model the high-order dependencies across multiple ST-scales through adaptive hypergraph modeling. Specifically, we introduce a Spatial-Temporal Pyramid Modeling (STPM) module to extract features at multiple ST-scales. Furthermore, we introduce an Adaptive Hypergraph Modeling (AHM) module that learns a sparse hypergraph to capture robust high-order dependencies among features. In addition, we interact with these features through tri-phase hypergraph propagation, which can comprehensively capture multi-scale spatial-temporal dynamics. Experimental results on six real-world MTS datasets demonstrate that ST-Hyper achieves the state-of-the-art performance, outperforming the best baselines with an average MAE reduction of 3.8\% and 6.8\% for long-term and short-term forecasting, respectively.
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Huawei Technologies Ltd.（华为技术有限公司）
Multivariate time series forecasting has been widely used in various practical scenarios. Recently, Transformer-based models have shown significant potential in forecasting tasks due to the capture of long-range dependencies. However, recent studies in the vision and NLP fields show that the role of attention modules is not clear, which can be replaced by other token aggregation operations. This paper investigates the contributions and deficiencies of attention mechanisms on the performance of time series forecasting. Specifically, we find that (1) attention is not necessary for capturing temp…
Multivariate time series (MTS) forecasting has attracted much attention in many intelligent applications. It is not a trivial task, as we need to consider both intra-variable dependencies and inter-variable dependencies. However, existing works are designed for specific scenarios, and require much domain knowledge and expert efforts, which is difficult to transfer between different scenarios. In this paper, we propose a scale-aware neural architecture search framework for MTS forecasting (SNAS4MTF). A multi-scale decomposition module transforms raw time series into multi-scale sub-series, whic…
Accurate and Efficient Multivariate Time Series Forecasting via Offline Clustering
State Key Laboratory of Complex & Critical Software Environment（复杂与关键软件环境国家重点实验室）；Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing（未来区块链与隐私计算北京先进创新中心）；Beihang University（北京航空航天大学）；HKGAI（Hong Kong University of Science and Technology）；Research Institute of Trustworthy Autonomous Systems（可信自主系统研究 institute）；Southern University of Science and Technology（南方科技大学）；Department of Data Science（数据科学系）
As large-scale time-series data can easily be found in real-world applications, multivariate time-series anomaly detection has played an essential role in diverse industries. It enables productivity improvement and maintenance cost reduction by preventing malfunctions and detecting anomalies based on time-series data. However, multivariate time-series anomaly detection is challenging because real-world time-series data exhibit complex temporal dependencies. For this task, it is crucial to learn a rich representation that effectively contains the nonlinear temporal dynamics of normal behavior. …
Revitalizing Multivariate Time Series Forecasting: Learnable Decomposition with Inter-Series Dependencies and Intra-Series Variations Modeling
University of Electronic Science and Technology of China（电子科技大学）；The Hong Kong Polytechnic University（香港理工大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Cambridge（剑桥大学）
Ister: Linear Transformer for Efficient Multivariate Time Series Forecasting
Hong Kong University of Science and Technology (Guangzhou)（香港理工大学（广州））；ShenZhen University（深圳大学）；Shenzhen Energy Group Co., Ltd.（深圳能源集团有限公司）；South China University of Technology（华南理工大学）
GinAR: An End-To-End Multivariate Time Series Forecasting Model Suitable for Variable Missing
University of Chinese Academy of Sciences（中国科学院大学）；School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）
VCformer: Variable Correlation Transformer with Inherent Lagged Correlation for Multivariate Time Series Forecasting
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机与软件学院）；National Engineering Laboratory for Big Data System Computing Technology, Shenzhen University（深圳大学国家大数据系统计算技术工程实验室）
The multivariate time series forecasting has attracted more and more attention because of its vital role in different fields in the real world, such as finance, traffic, and weather. In recent years, many research efforts have been proposed for forecasting multivariate time series. Although some previous work considers the interdependencies among different variables in the same timestamp, existing work overlooks the inter-connections between different variables at different time stamps. In this paper, we propose a simple yet efficient instance-wise graph-based framework to utilize the inter-de…
Modeling inter-dependencies between time-series is the key to achieve high performance in anomaly detection for multivariate time-series data. The de-facto solution to model the dependencies is to feed the data into a recurrent neural network (RNN). However, the fully connected network structure underneath the RNN (either GRU or LSTM) assumes a static and complete dependency graph between time-series, which may not hold in many real-world applications. To alleviate this assumption, we propose a dynamic bipartite graph structure to encode the inter-dependencies between time-series. More concret…
Radiant First Research Limited（Radiant First研究有限公司）；The University of Hong Kong（香港大学）；Koi Investment Partners International Limited（Koi国际投资伙伴有限公司）
A reliable and efficient representation of multivariate time series is crucial in various downstream machine learning tasks. In multivariate time series forecasting, each variable depends on its historical values and there are inter-dependencies among variables as well. Models have to be designed to capture both intra- and inter-relationships among the time series. To move towards this goal, we propose the Time Series Attention Transformer (TSAT) for multivariate time series representation learning. Using TSAT, we represent both temporal information and inter-dependencies of multivariate time …
WaveRoRA: Wavelet Rotary Route Attention for Multivariate Time Series Forecasting
School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications（北京邮电大学计算机学院（国家示范性软件学院））；School of Computer Science and Engineering, University of Texas at Arlington（德克萨斯大学阿灵顿分校计算机科学与工程学院）
University of Electronic Science and Technology of China（电子科技大学）；Aalborg University（奥尔堡大学）
Monitoring complex systems results in massive multivariate time series data, and anomaly detection of these data is very important to maintain the normal operation of the systems. Despite the recent emergence of a large number of anomaly detection algorithms for multivariate time series, most of them ignore the correlation modeling among multivariate, which can often lead to poor anomaly detection results. In this work, we propose a novel anomaly detection model for multivariate time series with \underline{HI}gh-order \underline{F}eature \underline{I}nteractions (HIFI). More specifically, HIFI…
FMamba: Mamba based on Fast-attention for Multivariate Time-series Forecasting
Institute of Advanced Technology, USTC（中国科学技术大学先进技术研究院）；Department of Automation, USTC（中国科学技术大学自动化系）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
Pennsylvania State University（宾夕法尼亚州立大学）；IBM T.J. Watson（IBM托马斯·J·沃森研究中心）
Multivariate time series (MTS) forecasting is widely used in various domains, such as meteorology and traffic. Due to limitations on data collection, transmission, and storage, real-world MTS data usually contains missing values, making it infeasible to apply existing MTS forecasting models such as linear regression and recurrent neural networks. Though many efforts have been devoted to this problem, most of them solely rely on local dependencies for imputing missing values, which ignores global temporal dynamics. Local dependencies/patterns would become less useful when the missing ratio is h…
Datasets with hundreds of variables and many missing values are commonplace. In this setting, it is both statistically and computationally challenging to detect true predictive relationships between variables and also to suppress false positives. This paper proposes an approach that combines probabilistic programming, information theory, and non-parametric Bayes. It shows how to use Bayesian non-parametric modeling to (i) build an ensemble of joint probability models for all the variables; (ii) efficiently detect marginal independencies; and (iii) estimate the conditional mutual information be…
IBM Thomas J. Watson Research Center（IBM托马斯·J·沃森研究中心）
Discovering temporal lagged and inter-dependencies in multivariate time series data is an important task. However, in many real-world applications, such as commercial cloud management, manufacturing predictive maintenance, and portfolios performance analysis, such dependencies can be non-linear and time-variant, which makes it more challenging to extract such dependencies through traditional methods such as Granger causality or clustering. In this work, we present a novel deep learning model that uses multiple layers of customized gated recurrent units (GRUs) for discovering both time lagged b…
