Multivariate time series prediction has applications in a wide variety of domains and is considered to be a very challenging task, especially when the variables have correlations and exhibit complex temporal patterns, such as seasonality and trend. Many existing methods suffer from strong statistical assumptions, numerical issues with high dimensionality, manual feature engineering efforts, and scalability. In this work, we present a novel deep learning architecture, known as Temporal Tensor Transformation Network, which transforms the original multivariate time series into a higher order of tensor through the proposed Temporal-Slicing Stack Transformation. This yields a new representation of the original multivariate time series, which enables the convolution kernel to extract complex and non-linear features as well as variable interactional signals from a relatively large temporal region. Experimental results show that Temporal Tensor Transformation Network outperforms several state-of-the-art methods on window-based predictions across various tasks. The proposed architecture also demonstrates robust prediction performance through an extensive sensitivity analysis.
Time series prediction has been a long-standing research topic and an essential application in many domains. Modern time series collected from sensor networks (e.g., energy consumption and traffic flow) are often large-scale and incomplete with considerable corruption and missing values, making it difficult to perform accurate predictions. In this paper, we propose a low-rank autoregressive tensor completion (LATC) framework to model multivariate time series data. The key of LATC is to transform the original multivariate time series matrix (e.g., sensor$\times$time point) to a third-order tens…
Point process modeling is gaining increasing attention, as point process type data are emerging in numerous scientific applications. In this article, motivated by a neuronal spike trains study, we propose a novel point process regression model, where both the response and the predictor can be a high-dimensional point process. We model the predictor effects through the conditional intensities using a set of basis transferring functions in a convolutional fashion. We organize the corresponding transferring coefficients in the form of a three-way tensor, then impose the low-rank, sparsity, and su…
Classical moment based change point tests like the cusum test are very powerful in case of Gaussian time series with one change point but behave poorly under heavy tailed distributions and corrupted data. A new class of robust change point tests based on cusum statistics of robustly transformed observations is proposed. This framework is quite flexible, depending on the used transformation one can detect for instance changes in the mean, scale or dependence of a possibly multivariate time series. Simulations indicate that this approach is very powerful in detecting changes in the marginal vari…
Ecole Normale Supérieure de Cachan（卡尚高等师范学院）；CNRS（法国国家科学研究中心）；Université Paris-Saclay（巴黎-萨克雷大学）；Université Paris 13（巴黎第十三大学）
This paper introduces a new multivariate convolutional sparse coding based on tensor algebra with a general model enforcing both element-wise sparsity and low-rankness of the activations tensors. By using the CP decomposition, this model achieves a significantly more efficient encoding of the multivariate signal-particularly in the high order/ dimension setting-resulting in better performance. We prove that our model is closely related to the Kruskal tensor regression problem, offering interesting theoretical guarantees to our setting. Furthermore, we provide an efficient optimization algorith…
Learning the multivariate distribution of data is a core challenge in statistics and machine learning. Traditional methods aim for the probability density function (PDF) and are limited by the curse of dimensionality. Modern neural methods are mostly based on black-box models, lacking identifiability guarantees. In this work, we aim to learn multivariate cumulative distribution functions (CDFs), as they can handle mixed random variables, allow efficient box probability evaluation, and have the potential to overcome local sample scarcity owing to their cumulative nature. We show that any grid s…
On the one hand, artificial neural networks (ANNs) are commonly labelled as black-boxes, lacking interpretability; an issue that hinders human understanding of ANNs' behaviors. A need exists to generate a meaningful sequential logic of the ANN for interpreting a production process of a specific output. On the other hand, decision trees exhibit better interpretability and expressive power due to their representation language and the existence of efficient algorithms to transform the trees into rules. However, growing a decision tree based on the available data could produce larger than necessar…
Spatiotemporal traffic data (e.g., link speed/flow) collected from sensor networks can be organized as multivariate time series with additional spatial attributes. A crucial task in analyzing such data is to identify and detect anomalous observations and events from the data with complex spatial and temporal dependencies. Robust Principal Component Analysis (RPCA) is a widely used tool for anomaly detection. However, the traditional RPCA purely relies on the global low-rank assumption while ignoring the local temporal correlations. In light of this, this study proposes a Hankel-structured tens…
Chair of Statistics, University of Göttingen, Germany（统计学系，哥廷根大学）；Campus Institute Data Science, University of Göttingen, Germany（数据科学校园研究所，哥廷根大学）；Department of Medical Bioinformatics, University Medical Center Göttingen, Germany（医学生物信息学系，哥廷根大学医学中心）
University of Virginia（弗吉尼亚大学）；University of Minnesota（明尼苏达大学）；Amazon（亚马逊）
Effective non-parametric density estimation is a key challenge in high-dimensional multivariate data analysis. In this paper,we propose a novel approach that builds upon tensor factorization tools. Any multivariate density can be represented by its characteristic function, via the Fourier transform. If the sought density is compactly supported, then its characteristic function can be approximated, within controllable error, by a finite tensor of leading Fourier coefficients, whose size de-pends on the smoothness of the underlying density. This tensor can be naturally estimated from observed re…
Weifang University（潍坊学院）；The University of Hong Kong（香港大学）；Central China Normal University（华中师范大学）
One of the key problems in tensor completion is the number of uniformly random sample entries required for recovery guarantee. The main aim of this paper is to study $n_1 \times n_2 \times n_3$ third-order tensor completion based on transformed tensor singular value decomposition, and provide a bound on the number of required sample entries. Our approach is to make use of the multi-rank of the underlying tensor instead of its tubal rank in the bound. In numerical experiments on synthetic and imaging data sets, we demonstrate the effectiveness of our proposed bound for the number of sample entr…
University of Florida Informatics Institute（佛罗里达大学信息学研究所）；University of Florida（佛罗里达大学）
Network data are commonly collected in a variety of applications, representing either directly measured or statistically inferred connections between features of interest. In an increasing number of domains, these networks are collected over time, such as interactions between users of a social media platform on different days, or across multiple subjects, such as in multi-subject studies of brain connectivity. When analyzing multiple large networks, dimensionality reduction techniques are often used to embed networks in a more tractable low-dimensional space. To this end, we develop a framewor…
We propose a continuous neural network architecture, termed Explainable Tensorized Neural Ordinary Differential Equations (ETN-ODE), for multi-step time series prediction at arbitrary time points. Unlike the existing approaches, which mainly handle univariate time series for multi-step prediction or multivariate time series for single-step prediction, ETN-ODE could model multivariate time series for arbitrary-step prediction. In addition, it enjoys a tandem attention, w.r.t. temporal attention and variable attention, being able to provide explainable insights into the data. Specifically, ETN-O…
Deep Multi-Manifold Transformation Based Multivariate Time Series Fault Detection
School of Information and Electrical Engineering, Hangzhou City University（杭州城市大学信息与电子工程学院）；Academy of Edge Intelligence, Hangzhou City University（杭州城市大学边缘智能研究院）；College of Information Engineering, Zhejiang University of Technology（浙江工业大学信息工程学院）；Institute of Cyber-Systems and Control, Zhejiang University（浙江大学网络系统与控制研究所）；Centre for Artificial Intelligence and Robotics (CAIR), HKISI-CAS（人工智能与机器人中心（CAIR））；State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA)（多模态人工智能系统国家重点实验室（MAIS））；School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)（中国科学院大学人工智能学院）
