In this paper, we propose a dimensionless anomaly detection method for multivariate streams. Our method is independent of the unit of measurement for the different stream channels, therefore dimensionless. We first propose the variance norm, a generalisation of Mahalanobis distance to handle infinite-dimensional feature space and singular empirical covariance matrix rigorously. We then combine the variance norm with the path signature, an infinite collection of iterated integrals that provide global features of streams, to propose SigMahaKNN, a method for anomaly detection on (multivariate) streams. We show that SigMahaKNN is invariant to stream reparametrisation, stream concatenation and has a graded discrimination power depending on the truncation level of the path signature. We implement SigMahaKNN as an open-source software, and perform extensive numerical experiments, showing significantly improved anomaly detection on streams compared to isolation forest and local outlier factors in applications ranging from language analysis, hand-writing analysis, ship movement paths analysis and univariate time-series analysis.
Many events occur in the world. Some event types are stochastically excited or inhibited---in the sense of having their probabilities elevated or decreased---by patterns in the sequence of previous events. Discovering such patterns can help us predict which type of event will happen next and when. We model streams of discrete events in continuous time, by constructing a neurally self-modulating multivariate point process in which the intensities of multiple event types evolve according to a novel continuous-time LSTM. This generative model allows past events to influence the future in complex …
RED-F: Reconstruction-Elimination based Dual-stream Contrastive Forecasting for Multivariate Time Series Anomaly Prediction
School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications（计算机学院（国家级试点软件工程学院），北京邮电大学）；China Telecom Research Institute Beijing（中国电信研究院北京分院）；Department of Computer Science, Missouri University of Science and Technology（计算机学院，密苏里科技大学）
DSAT-HD: Dual-Stream Adaptive Transformer with Hybrid Decomposition for Multivariate Time Series Forecasting
Computer Science and Technology（计算机科学与技术）；Harbin Engineering University（哈尔滨工程大学）；Traditional Chinese Medicine Data Center（传统中医数据中心）；China Academy of Chinese Medical Sciences（中国中医科学院）
Università della Svizzera italiana（瑞士意大利语大学）；Politecnico di Milano（米兰理工大学）；University of Exeter（埃克塞特大学）
Graph representations offer powerful and intuitive ways to describe data in a multitude of application domains. Here, we consider stochastic processes generating graphs and propose a methodology for detecting changes in stationarity of such processes. The methodology is general and considers a process generating attributed graphs with a variable number of vertices/edges, without the need to assume one-to-one correspondence between vertices at different time steps. The methodology acts by embedding every graph of the stream into a vector domain, where a conventional multivariate change detectio…
Recently, digital payment systems have significantly changed people's lifestyles. New challenges have surfaced in monitoring and guaranteeing the integrity of payment processing systems. One important task is to predict the future transaction statistics of each merchant. These predictions can thus be used to steer other tasks, ranging from fraud detection to recommendation. This problem is challenging as we need to predict not only multivariate time series but also multi-steps into the future. In this work, we propose a multi-stream RNN model for multi-step merchant transaction predictions tai…
Hydrological storm events are a primary driver for transporting water quality constituents such as turbidity, suspended sediments and nutrients. Analyzing the concentration (C) of these water quality constituents in response to increased streamflow discharge (Q), particularly when monitored at high temporal resolution during a hydrological event, helps to characterize the dynamics and flux of such constituents. A conventional approach to storm event analysis is to reduce the C-Q time series to two-dimensional (2-D) hysteresis loops and analyze these 2-D patterns. While effective and informativ…
University of North Carolina Charlotte（北卡罗来纳大学夏洛特分校）
The effect of bias on hypothesis formation is characterized for an automated data-driven projection pursuit neural network to extract and select features for binary classification of data streams. This intelligent exploratory process partitions a complete vector state space into disjoint subspaces to create working hypotheses quantified by similarities and differences observed between two groups of labeled data streams. Data streams are typically time sequenced, and may exhibit complex spatio-temporal patterns. For example, given atomic trajectories from molecular dynamics simulation, the mach…
University of Warwick（华威大学）；Alan Turing Institute（艾伦·图灵研究所）；University of Oxford（牛津大学）；CSIRO’s Data61（澳大利亚联邦科学与工业研究组织数据61中心）
Distribution regression refers to the supervised learning problem where labels are only available for groups of inputs instead of individual inputs. In this paper, we develop a rigorous mathematical framework for distribution regression where inputs are complex data streams. Leveraging properties of the expected signature and a recent signature kernel trick for sequential data from stochastic analysis, we introduce two new learning techniques, one feature-based and the other kernel-based. Each is suited to a different data regime in terms of the number of data streams and the dimensionality of…
National University of Singapore（新加坡国立大学）；University of California, Berkeley（加州大学伯克利分校）；ST Electronics (Info Security) Pte Ltd（新科电子（信息安全）私人有限公司）
The prevalence of networked sensors and actuators in many real-world systems such as smart buildings, factories, power plants, and data centers generate substantial amounts of multivariate time series data for these systems. The rich sensor data can be continuously monitored for intrusion events through anomaly detection. However, conventional threshold-based anomaly detection methods are inadequate due to the dynamic complexities of these systems, while supervised machine learning methods are unable to exploit the large amounts of data due to the lack of labeled data. On the other hand, curre…
In this paper we describe fast Bayesian statistical analysis of vector positive-valued time series, with application to interesting financial data streams. We discuss a flexible level correlated model (LCM) framework for building hierarchical models for vector positive-valued time series. The LCM allows us to combine marginal gamma distributions for the positive-valued component responses, while accounting for association among the components at a latent level. We use integrated nested Laplace approximation (INLA) for fast approximate Bayesian modeling via the R-INLA package, building custom f…
In data science, vector autoregression (VAR) models are popular in modeling multivariate time series in the environmental sciences and other applications. However, these models are computationally complex with the number of parameters scaling quadratically with the number of time series. In this work, we propose a so-called neighborhood vector autoregression (NVAR) model to efficiently analyze large-dimensional multivariate time series. We assume that the time series have underlying neighborhood relationships, e.g., spatial or network, among them based on the inherent setting of the problem. W…
ChronoGraph: A Real-World Graph-Based Multivariate Time Series Dataset
Bitdefender；University of Bucharest（布加勒斯特大学）；University of Stuttgart（斯图加特大学）；International Max Planck Research School for Intelligent Systems（智能系统国际马克斯·普朗克研究学校）
National University of Singapore（新加坡国立大学）；ST Electronics (Info Security) Pte Ltd.（新科电子（信息安全）私人有限公司）
Today's Cyber-Physical Systems (CPSs) are large, complex, and affixed with networked sensors and actuators that are targets for cyber-attacks. Conventional detection techniques are unable to deal with the increasingly dynamic and complex nature of the CPSs. On the other hand, the networked sensors and actuators generate large amounts of data streams that can be continuously monitored for intrusion events. Unsupervised machine learning techniques can be used to model the system behaviour and classify deviant behaviours as possible attacks. In this work, we proposed a novel Generative Adversaria…
Modeling inter-dependencies between time-series is the key to achieve high performance in anomaly detection for multivariate time-series data. The de-facto solution to model the dependencies is to feed the data into a recurrent neural network (RNN). However, the fully connected network structure underneath the RNN (either GRU or LSTM) assumes a static and complete dependency graph between time-series, which may not hold in many real-world applications. To alleviate this assumption, we propose a dynamic bipartite graph structure to encode the inter-dependencies between time-series. More concret…
Combining Statistical Features and Deep Encodings for Rehearsal-Based Class-Incremental Time Series Classification
atlanTTic – ICLAB, Universidade de Vigo（atlanTTic–ICLAB，维戈大学）；Centro Tecnolóxico de Telecomunicacións de Galicia (GRADIANT)（加利西亚电信技术中心（GRADIANT））；Universidade de Vigo（维戈大学）
In this work, we aim to provide a new and efficient recursive detection method for temporarily monitored signals. Motivated by the case of the propagation of an event over a field of sensors, we assumed that the change in the statistical properties in the monitored signals can only be temporary. Unfortunately, to our best knowledge, existing recursive and simple detection techniques such as the ones based on the cumulative sum (CUSUM) do not consider the temporary aspect of the change in a multivariate time series. In this paper, we propose a novel simple and efficient sequential detection alg…
In this paper, we introduce Masked Anomaly Detection (MAD), a general self-supervised learning task for multivariate time series anomaly detection. With the increasing availability of sensor data from industrial systems, being able to detecting anomalies from streams of multivariate time series data is of significant importance. Given the scarcity of anomalies in real-world applications, the majority of literature has been focusing on modeling normality. The learned normal representations can empower anomaly detection as the model has learned to capture certain key underlying data regularities…
LLmFPCA-detect: LLM-powered Multivariate Functional PCA for Anomaly Detection in Sparse Longitudinal Texts
H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology（乔治亚理工学院H.Milton Stewart工业与系统工程学院）；AT&T Chief Data Office（AT&T首席数据办公室）；Department of Data Sciences and Operations, Marshall School of Business, University of Southern California（南加州大学马歇尔商学院数据科学与运营学院）
Time series forecasting (TSF) is one of the most important tasks in data science given the fact that accurate time series (TS) predictive models play a major role across a wide variety of domains including finance, transportation, health care, and power systems. Real-world utilization of machine learning (ML) typically involves (pre-)training models on collected, historical data and then applying them to unseen data points. However, in real-world applications, time series data streams are usually non-stationary and trained ML models usually, over time, face the problem of data or concept drift…
This paper introduces the Hawkes skeleton and the Hawkes graph. These objects summarize the branching structure of a multivariate Hawkes point process in a compact, yet meaningful way. We demonstrate how graph-theoretic vocabulary (`ancestor sets', `parent sets', `connectivity', `walks', `walk weights', ...) is very convenient for the discussion of multivariate Hawkes processes. For example, we reformulate the classic eigenvalue-based subcriticality criterion of multitype branching processes in graph terms. Next to these more terminological contributions, we show how the graph view may be used…
Distributed acoustic sensing technology is increasingly being used to support production and well management within the oil and gas sector, for example to improve flow monitoring and production profiling. This sensing technology is capable of recording substantial data volumes at multiple depths within an oil well, giving unprecedented insights into production behaviour. However the technology is also prone to recording periods of anomalous behaviour, where the same physical features are concurrently observed at multiple depths. Such features are called `stripes' and are undesirable, detriment…
Many real-world objects can be modeled as a stream of events on the nodes of a graph. In this paper, we propose a class of graphical event models named temporal point process graphical models for representing the temporal dependencies among different components of a multivariate point process. In our model, the intensity of an event stream can depend on the historical events in a nonlinear way. We provide a procedure that allows us to estimate the parameters in the model with a convex loss function in the high-dimensional setting. For the approximation error introduced during the implementatio…
Modeling Time-evolving Causality over Data Streams
SANKEN, Osaka University（SANKEN，大阪大学）；Institute for Clarity in Documentation（文档清晰研究所）；Inria Paris-Rocquencourt（巴黎-鲁克勒特研究所）；Rajiv Gandhi University（拉贾·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒姆研究实验室）
In this paper, we present a nonparametric estimation procedure for the multivariate Hawkes point process. The timeline is cut into bins and -- for each component process -- the number of points in each bin is counted. The distribution of the resulting "bin-count sequences" can be approximated by an integer-valued autoregressive model known as the (multivariate) INAR($p$) model. We represent the INAR($p$) model as a standard vector-valued linear autoregressive time series with white-noise innovations (VAR($p$)). We establish consistency and asymptotic normality for conditional least-squares est…
In recent years, there has been an ever increasing amount of multivariate time series (MTS) data in various domains, typically generated by a large family of sensors such as wearable devices. This has led to the development of novel learning methods on MTS data, with deep learning models dominating the most recent advancements. Prior literature has primarily focused on designing new network architectures for modeling temporal dependencies within MTS. However, a less studied challenge is associated with high dimensionality of MTS data. In this paper, we propose a novel neural component, namely …
Functional brain imaging through electroencephalography (EEG) relies upon the analysis and interpretation of high-dimensional, spatially organized time series. We propose to represent time-localized frequency domain characterizations of EEG data as region-referenced functional data. This representation is coupled with a hierarchical modeling approach to multivariate functional observations. Within this familiar setting, we discuss how several prior models relate to structural assumptions about multivariate covariance operators. An overarching modeling framework, based on infinite factorial dec…
Multivariate Time Series Clustering for Environmental State Characterization of Ground-Based Gravitational-Wave Detectors
Computer Science \& Engineering University of California, Riverside Riverside, CA, USA；Physics University of Dallas Dallas, TX, USA；Physics \& Astronomy University of California, Riverside Riverside, CA, USA；LIGO Laboratory LIGO Livingston Observatory Livingston, LA, USA
Syndromic surveillance systems continuously monitor multiple pre-diagnostic daily streams of indicators from different regions with the aim of early detection of disease outbreaks. The main objective of these systems is to detect outbreaks hours or days before the clinical and laboratory confirmation. The type of data that is being generated via these systems is usually multivariate and seasonal with spatial and temporal dimensions. The algorithm What's Strange About Recent Events (WSARE) is the state-of-the-art method for such problems. It exhaustively searches for contrast sets in the multiv…
