This paper presents an extensive empirical study on the integration of dimensionality reduction techniques with advanced unsupervised time series anomaly detection models, focusing on the MUTANT and Anomaly-Transformer models. The study involves a comprehensive evaluation across three different datasets: MSL, SMAP, and SWaT. Each dataset poses unique challenges, allowing for a robust assessment of the models' capabilities in varied contexts. The dimensionality reduction techniques examined include PCA, UMAP, Random Projection, and t-SNE, each offering distinct advantages in simplifying high-dimensional data. Our findings reveal that dimensionality reduction not only aids in reducing computational complexity but also significantly enhances anomaly detection performance in certain scenarios. Moreover, a remarkable reduction in training times was observed, with reductions by approximately 300\% and 650\% when dimensionality was halved and minimized to the lowest dimensions, respectively. This efficiency gain underscores the dual benefit of dimensionality reduction in both performance enhancement and operational efficiency. The MUTANT model exhibits notable adaptability, especially with UMAP reduction, while the Anomaly-Transformer demonstrates versatility across various reduction techniques. These insights provide a deeper understanding of the synergistic effects of dimensionality reduction and anomaly detection, contributing valuable perspectives to the field of time series analysis. The study underscores the importance of selecting appropriate dimensionality reduction strategies based on specific model requirements and dataset characteristics, paving the way for more efficient, accurate, and scalable solutions in anomaly detection.
Anomaly detection is a field of intense research. Identifying low probability events in data/images is a challenging problem given the high-dimensionality of the data, especially when no (or little) information about the anomaly is available a priori. While plenty of methods are available, the vast majority of them do not scale well to large datasets and require the choice of some (very often critical) hyperparameters. Therefore, unsupervised and computationally efficient detection methods become strictly necessary. We propose an unsupervised method for detecting anomalies and changes in remot…
Multivariate de Bruijn Graphs: A Symbolic Graph Framework for Time Series Forecasting
Department of Computer Science, Indiana University Bloomington, IN, USA（印第安纳大学布卢明顿分校计算机科学系）；College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar（哈马德·本·卡西姆大学科学与工程学院）；Department of Computer Science, University of Illinois Springfield, IL, USA（伊利诺伊大学斯普林菲尔德分校计算机科学系）
Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysis
Institute of Computing Technology, CAS（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；Tsinghua University（清华大学）；School of Computer Science and Engineering, The University of New South Wales（新南威尔士大学计算机科学与工程学院）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）；Department of Computer Science, Aalborg University（奥尔堡大学计算机科学系）
Pennsylvania State University（宾夕法尼亚州立大学）；IBM T.J. Watson（IBM托马斯·J·沃森研究中心）
Multivariate time series (MTS) forecasting is widely used in various domains, such as meteorology and traffic. Due to limitations on data collection, transmission, and storage, real-world MTS data usually contains missing values, making it infeasible to apply existing MTS forecasting models such as linear regression and recurrent neural networks. Though many efforts have been devoted to this problem, most of them solely rely on local dependencies for imputing missing values, which ignores global temporal dynamics. Local dependencies/patterns would become less useful when the missing ratio is h…
Bogazici University（博阿齐奇大学）；Mississippi State University（密西西比州立大学）
This paper gives a review and synthesis of methods of evaluating dimensionality reduction techniques. Particular attention is paid to rank-order neighborhood evaluation metrics. A framework is created for exploring dimensionality reduction quality through visualization. An associated toolkit is implemented in R. The toolkit includes scatter plots, heat maps, loess smoothing, and performance lift diagrams. The overall rationale is to help researchers compare dimensionality reduction techniques and use visual insights to help select and improve techniques. Examples are given for dimensionality r…
Federal Office for Information Security（德国联邦信息安全办公室）
A new method to derive Multivariate Quadratic equation systems (MQ) for the input and output bit variables of a cryptographic S-box from its algebraic expressions with the aid of the computer mathematics software system SageMath is presented. We consolidate the deficiency of previously presented MQ metrics, supposed to quantify the resistance of S-boxes against algebraic attacks.
University of California, Davis（加利福尼亚大学戴维斯分校）；Kobe University（神户大学）；RIKEN R-CCS（日本理化学研究所计算科学研究中心）
Data-driven problem solving in many real-world applications involves analysis of time-dependent multivariate data, for which dimensionality reduction (DR) methods are often used to uncover the intrinsic structure and features of the data. However, DR is usually applied to a subset of data that is either single-time-point multivariate or univariate time-series, resulting in the need to manually examine and correlate the DR results out of different data subsets. When the number of dimensions is large either in terms of the number of time points or attributes, this manual task becomes too tedious…
Time series sequence prediction and modelling has proven to be a challenging endeavor in real world datasets. Two key issues are the multi-dimensionality of data and the interaction of independent dimensions forming a latent output signal, as well as the representation of multi-dimensional temporal data inside of a predictive model. This paper proposes a multi-branch deep neural network approach to tackling the aforementioned problems by modelling a latent state vector representation of data windows through the use of a recurrent autoencoder branch and subsequently feeding the trained latent v…
Navigating High-Dimensional Backstage: A Guide for Exploring Literature for the Reliable Use of Dimensionality Reduction
Seoul National University（首尔国立大学）；UNIST（全南大学）；University of California, Davis（加州大学戴维斯分校）；Newcastle University（新castle大学）；POSTECH；Linköping University（林肯大学）
We introduce Contrastive Multivariate Singular Spectrum Analysis, a novel unsupervised method for dimensionality reduction and signal decomposition of time series data. By utilizing an appropriate background dataset, the method transforms a target time series dataset in a way that evinces the sub-signals that are enhanced in the target dataset, as opposed to only those that account for the greatest variance. This shifts the goal from finding signals that explain the most variance to signals that matter the most to the analyst. We demonstrate our method on an illustrative synthetic example, as …
There are different problems for resolution of complex LC-MS or GC-MS data, such as the existence of embedded chromatographic peaks, continuum background and overlapping in mass channels for different components. These problems cause rotational ambiguity in recovered profiles calculated using multivariate curve resolution (MCR) methods. Since mass spectra are sparse in nature, sparsity has been proposed recently as a constraint in MCR methods for analyzing LC-MS data. There are different ways for implementation of the sparsity constraint, and majority of methods rely on imposing a penalty base…
Dimensionality reduction techniques map data represented on higher dimensions onto lower dimensions with varying degrees of information loss. Graph dimensionality reduction techniques adopt the same principle of providing latent representations of the graph structure with minor adaptations to the output representations along with the input data. There exist several cutting edge techniques that are efficient at generating embeddings from graph data and projecting them onto low dimensional latent spaces. Due to variations in the operational philosophy, the benefits of a particular graph dimensio…
Ansbach University of Applied Sciences（安斯巴赫应用科学大学）
Multivariate signal processing is often based on dimensionality reduction techniques. We propose a new method, Dynamical Component Analysis (DyCA), leading to a classification of the underlying dynamics and - for a certain type of dynamics - to a signal subspace representing the dynamics of the data. In this paper the algorithm is derived leading to a generalized eigenvalue problem of correlation matrices. The application of the DyCA on high-dimensional chaotic signals is presented both for simulated data as well as real EEG data of epileptic seizures.
Exploring Spiking Neural Networks for Binary Classification in Multivariate Time Series at the Edge
Computer Science and Engineering（计算机科学与工程）；Louisiana State University（路易斯安那州立大学）；Physics（物理学）；Oak Ridge National Laboratory（橡树岭国家实验室）；Electrical Engineering and Computer Science（电气工程与计算机科学）；The University of Tennessee Knoxville（田纳西大学基洛纳分校）；Computer Science and Mathematics（计算机科学与数学）；Electrification and Energy Infrastructures（能源基础设施）
Multivariate time series prediction has applications in a wide variety of domains and is considered to be a very challenging task, especially when the variables have correlations and exhibit complex temporal patterns, such as seasonality and trend. Many existing methods suffer from strong statistical assumptions, numerical issues with high dimensionality, manual feature engineering efforts, and scalability. In this work, we present a novel deep learning architecture, known as Temporal Tensor Transformation Network, which transforms the original multivariate time series into a higher order of t…
Exploring Quantum Control Landscape and Solution Space Complexity through Dimensionality Reduction & Optimization Algorithms
School of Computer Science, University College Dublin（计算机科学学院，都柏林大学学院）；Centre for Quantum Engineering, Science, and Technology, University College Dublin（量子工程、科学与技术中心，都柏林大学学院）；School of Physics, University College Dublin（物理学院，都柏林大学学院）
Explainable AI for Multivariate Time Series Pattern Exploration: Latent Space Visual Analytics with Temporal Fusion Transformer and Variational Autoencoders in Power Grid Event Diagnosis
University of Electronic Science and Technology of China（电子科技大学）
Scientists in many fields have the common and basic need of dimensionality reduction: visualizing the underlying structure of the massive multivariate data in a low-dimensional space. However, many dimensionality reduction methods confront the so-called "crowding problem" that clusters tend to overlap with each other in the embedding. Previously, researchers expect to avoid that problem and seek to make clusters maximally separated in the embedding. However, the proposed in-tree (IT) based method, called IT-map, allows clusters in the embedding to be locally overlapped, while seeking to make t…
Multivariate time series (MTS) have become increasingly common in healthcare domains where human vital signs and laboratory results are collected for predictive diagnosis. Recently, there have been increasing efforts to visualize healthcare MTS data based on star charts or parallel coordinates. However, such techniques might not be ideal for visualizing a large MTS dataset, since it is difficult to obtain insights or interpretations due to the inherent high dimensionality of MTS. In this paper, we propose 'm-TSNE': a simple and novel framework to visualize high-dimensional MTS data by projecti…
Improving age prediction: Utilizing LSTM-based dynamic forecasting for data augmentation in multivariate time series analysis
Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS): Georgia State University, Georgia Institute of Technology, Emory University（转化神经影像与数据科学三机构中心（TReNDS）：佐治亚州立大学、佐治亚理工学院、埃默里大学）
Causal Discovery with Inverted Self-attention for Multivariate Time Series
School of Computer Science, University of Technology Sydney（悉尼科技大学计算机科学学院）；Technical University of Denmark（丹麦技术大学）；TianFu YongXing Laboratory（天府永兴实验室）；Faculty of Data Science, City University of Macau（澳门城市大学数据科学学院）
An emerging way of tackling the dimensionality issues arising in the modeling of a multivariate process is to assume that the inherent data structure can be captured by a graph. Nevertheless, though state-of-the-art graph-based methods have been successful for many learning tasks, they do not consider time-evolving signals and thus are not suitable for prediction. Based on the recently introduced joint stationarity framework for time-vertex processes, this letter considers multivariate models that exploit the graph topology so as to facilitate the prediction. The resulting method yields simila…
Cluster interpretation after dimensionality reduction (DR) is a ubiquitous part of exploring multidimensional datasets. DR results are frequently represented by scatterplots, where spatial proximity encodes similarity among data samples. In the literature, techniques support the understanding of scatterplots' organization by visualizing the importance of the features for cluster definition with layout enrichment strategies. However, current approaches usually focus on global information, hampering the analysis whenever the focus is to understand the differences among clusters. Thus, this paper…
Deep Variational Multivariate Information Bottleneck -- A Framework for Variational Losses
Department of Physics Emory University（埃默里大学物理系）；Departments of Physics and Biology Initiative in Theory and Modeling of Living Systems Emory University（埃默里大学物理与生物学理论与活体系统建模研究所）；Georgia Institute of Technology（佐治亚理工学院）
Florida State University（佛罗里达州立大学）；Peking University（北京大学）
Modern high-dimensional methods often adopt the "bet on sparsity" principle, while in supervised multivariate learning statisticians may face "dense" problems with a large number of nonzero coefficients. This paper proposes a novel clustered reduced-rank learning (CRL) framework that imposes two joint matrix regularizations to automatically group the features in constructing predictive factors. CRL is more interpretable than low-rank modeling and relaxes the stringent sparsity assumption in variable selection. In this paper, new information-theoretical limits are presented to reveal the intrin…
University of Konstanz（康斯坦茨大学）；University of Stuttgart（斯图加特大学）
We present a technique to perform dimensionality reduction on data that is subject to uncertainty. Our method is a generalization of traditional principal component analysis (PCA) to multivariate probability distributions. In comparison to non-linear methods, linear dimensionality reduction techniques have the advantage that the characteristics of such probability distributions remain intact after projection. We derive a representation of the PCA sample covariance matrix that respects potential uncertainty in each of the inputs, building the mathematical foundation of our new method: uncertain…
Rey Juan Carlos University（胡安·卡洛斯国王大学）；University Hospital of Fuenlabrada（富恩拉夫拉达大学医院）；University Hospital of North-Norway（北挪威大学医院）；UiT The Arctic University of Norway（挪威北极大学）
The acquisition of Antimicrobial Multidrug Resistance (AMR) in patients admitted to the Intensive Care Units (ICU) is a major global concern. This study analyses data in the form of multivariate time series (MTS) from 3476 patients recorded at the ICU of University Hospital of Fuenlabrada (Madrid) from 2004 to 2020. 18\% of the patients acquired AMR during their stay in the ICU. The goal of this paper is an early prediction of the development of AMR. Towards that end, we leverage the time-series cluster kernel (TCK) to learn similarities between MTS. To evaluate the effectiveness of TCK as a k…
