We present a robust test for change-points in time series which is based on the two-sample Hodges-Lehmann estimator. We develop new limit theory for a class of statistics based on the two-sample U-quantile processes, in the case of short range dependent observations. Using this theory we can derive the asymptotic distribution of our test statistic under the null hypothesis. We study the finite sample properties of our test via a simulation study and compare the test with the classical CUSUM test and a test based on the Wilcoxon-Mann-Whitney statistic.
The Shift Operator Calculus for Stationary Time Series Analysis
National Institute of Advanced Studies, Manipal Academy of Higher Education（高等研究院，马匹尔高等教育学院）；Department of Mathematics, Indian Institute of Science（数学系，印度科学学院）
We describe a family of conservative statistical tests for independence of two autocorrelated time series. The series may take values in any sets, and one of them must be stationary. A user-specified function quantifying the association of a segment of the two series is compared to an ensemble obtained by time-shifting the stationary series -N to N steps. If the series are independent, the unshifted value is in the top m shifted values with probability at most m/(N+1). For large N, the probability approaches m/(2N+1). A conservative test rejects independence at significance α if the unshifted …
Temporal distributional shifts, with underlying dynamics changing over time, frequently occur in real-world time series and pose a fundamental challenge for deep neural networks (DNNs). In this paper, we propose a novel deep sequence model based on the Koopman theory for time series forecasting: Koopman Neural Forecaster (KNF) which leverages DNNs to learn the linear Koopman space and the coefficients of chosen measurement functions. KNF imposes appropriate inductive biases for improved robustness against distributional shifts, employing both a global operator to learn shared characteristics a…
We consider the distribution of the turning point location of time series modeled as the sum of deterministic trend plus random noise. If the variables are modeled by shifted exponentials, whose location parameters define the trend, we provide a formula for computing the distribution of the turning point location and consequently to estimate a confidence interval for the location. We test this formula in simulated data series having a trend with asymmetric minimum, investigating the coverage rate as a function of a bandwidth parameter. The method is applied to estimate the confidence interval …
University of Sherbrooke（舍布鲁克大学）；University of Quebec at Montreal（魁北克大学蒙特利尔分校）
We investigate the problem of discovering and modeling regime shifts in an ecosystem comprising multiple time series known as co-evolving time series. Regime shifts refer to the changing behaviors exhibited by series at different time intervals. Learning these changing behaviors is a key step toward time series forecasting. While advances have been made, existing methods suffer from one or more of the following shortcomings: (1) failure to take relationships between time series into consideration for discovering regimes in multiple time series; (2) lack of an effective approach that models tim…
This article studies estimation of a stationary autocovariance structure in the presence of an unknown number of mean shifts. Here, a Yule-Walker moment estimator for the autoregressive parameters in a dependent time series contaminated by mean shift changepoints is proposed and studied. The estimator is based on first order differences of the series and is proven consistent and asymptotically normal when the number of changepoints $m$ and the series length $N$ satisfies $m/N \rightarrow 0$ as $N \rightarrow \infty$
Time series often contain outliers and level shifts or structural changes. These unexpected events are of the utmost importance in fraud detection, as they may pinpoint suspicious transactions. The presence of such unusual events can easily mislead conventional time series analysis and yield erroneous conclusions. In this paper we provide a unified framework for detecting outliers and level shifts in short time series that may have a seasonal pattern. The approach combines ideas from the FastLTS algorithm for robust regression with alternating least squares. The double wedge plot is proposed, …
Two methods for detecting abrupt shifts in the variance, Integrated Cumulative Sum of Squares (ICSS) and Sequential Regime Shift Detector (SRSD), have been compared on both synthetic and observed time series. In Monte Carlo experiments, SRSD outperformed ICSS in the overwhelming majority of the modelled scenarios with different sequences of variance regimes. The SRSD advantage was particularly apparent in the case of outliers in the series. When tested on climatic time series, in most cases both methods detected the same change points in the longer series (252-787 monthly values). The only exc…
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；Harvard University (SEAS)（哈佛大学工程与应用科学学院）；Lunenfeld Tanenbaum Research Institute（卢嫩菲尔德-塔南鲍姆研究所）；The Hospital for Sick Children（病童医院）
Explanations of time series models are useful for high stakes applications like healthcare but have received little attention in machine learning literature. We propose FIT, a framework that evaluates the importance of observations for a multivariate time-series black-box model by quantifying the shift in the predictive distribution over time. FIT defines the importance of an observation based on its contribution to the distributional shift under a KL-divergence that contrasts the predictive distribution against a counterfactual where the rest of the features are unobserved. We also demonstrat…
Decomposing complex time series into trend, seasonality, and remainder components is an important task to facilitate time series anomaly detection and forecasting. Although numerous methods have been proposed, there are still many time series characteristics exhibiting in real-world data which are not addressed properly, including 1) ability to handle seasonality fluctuation and shift, and abrupt change in trend and reminder; 2) robustness on data with anomalies; 3) applicability on time series with long seasonality period. In the paper, we propose a novel and generic time series decomposition…
The monitoring and management of numerous and diverse time series data at Alibaba Group calls for an effective and scalable time series anomaly detection service. In this paper, we propose RobustTAD, a Robust Time series Anomaly Detection framework by integrating robust seasonal-trend decomposition and convolutional neural network for time series data. The seasonal-trend decomposition can effectively handle complicated patterns in time series, and meanwhile significantly simplifies the architecture of the neural network, which is an encoder-decoder architecture with skip connections. This arch…
A novel method for predicting time series is described and demonstrated. This method inputs time series data points and outputs multiple "spaghetti" functions from which predictions can be made. Spaghetti prediction has desirable properties that are not realized by classic autoregression, moving average, spline, Gaussian process, and other methods. It is particularly appropriate for short time series because it allows asymmetric prediction distributions and produces prediction functions which are robust in that they use multiple independent models.
This note outlines a method for clustering time series based on a statistical model in which volatility shifts at unobserved change-points. The model accommodates some classical stylized features of returns and its relation to GARCH is discussed. Clustering is performed using a probability metric evaluated between posterior distributions of the most recent change-point associated with each series. This implies series are grouped together at a given time if there is evidence the most recent shifts in their respective volatilities were coincident or closely timed. The clustering method is dynami…
This paper reviews recent developments of robust estimation in linear time series models, with short and long memory correlation structures, in the presence of additive outliers. Based on the manuscripts Fajardo et al. (2009) and Lévy-Leduc et al. (2011a), the emphasis in this paper is given in the following directions; the influence of additive outliers in the estimation of a time series, the asymptotic properties of a robust autocovariance function and a robust semiparametric estimation method of the fractional parameter d in ARFIMA(p, d, q) models. Some simulations are used to support the u…
A method is proposed to generate an optimal fit of a number of connected linear trend segments onto time-series data. To be able to efficiently handle many lines, the method employs a stochastic search procedure to determine optimal transition point locations. Traditional methods use exhaustive grid searches, which severely limit the scale of the problems for which they can be utilized. The proposed approach is tried against time series with severe noise to demonstrate its robustness, and then it is applied to real medical data as an illustrative example.
Univariate time series often take the form of a collection of curves observed sequentially over time. Examples of these include hourly ground-level ozone concentration curves. These curves can be viewed as a time series of functions observed at equally spaced intervals over a dense grid. Since functional time series may contain various types of outliers, we introduce a robust functional time series forecasting method to down-weigh the influence of outliers in forecasting. Through a robust principal component analysis based on projection pursuit, a time series of functions can be decomposed int…
In this work a robust clustering algorithm for stationary time series is proposed. The algorithm is based on the use of estimated spectral densities, which are considered as functional data, as the basic characteristic of stationary time series for clustering purposes. A robust algorithm for functional data is then applied to the set of spectral densities. Trimming techniques and restrictions on the scatter within groups reduce the effect of noise in the data and help to prevent the identification of spurious clusters. The procedure is tested in a simulation study, and is also applied to a rea…
Know-Center GmbH（Know-Center 有限公司）；Graz University of Technology（格拉茨技术大学）
Distance-based classification is among the most competitive classification methods for time series data. The most critical component of distance-based classification is the selected distance function. Past research has proposed various different distance metrics or measures dedicated to particular aspects of real-world time series data, yet there is an important aspect that has not been considered so far: Robustness against arbitrary data contamination. In this work, we propose a novel distance metric that is robust against arbitrarily "bad" contamination and has a worst-case computational com…
Periodicity detection is an important task in time series analysis, but still a challenging problem due to the diverse characteristics of time series data like abrupt trend change, outlier, noise, and especially block missing data. In this paper, we propose a robust and effective periodicity detection algorithm for time series with block missing data. We first design a robust trend filter to remove the interference of complicated trend patterns under missing data. Then, we propose a robust autocorrelation function (ACF) that can handle missing values and outliers effectively. We rigorously pro…
Dynamic time warping (DTW) is an effective dissimilarity measure in many time series applications. Despite its popularity, it is prone to noises and outliers, which leads to singularity problem and bias in the measurement. The time complexity of DTW is quadratic to the length of time series, making it inapplicable in real-time applications. In this paper, we propose a novel time series dissimilarity measure named RobustDTW to reduce the effects of noises and outliers. Specifically, the RobustDTW estimates the trend and optimizes the time warp in an alternating manner by utilizing our designed …
We present a new framework for robust estimation and inference on second-order stationary time series and random fields. This framework is based on the Generalized Method of Wavelet Moments which uses the wavelet variance to achieve parameter estimation for complex models. Using an M-estimator of the wavelet variance, this method can be made robust therefore allowing to estimate the parameters of a wide range of time series and spatial models when the data suffers from outliers or different forms of contamination. The paper presents a series of simulation studies as well as a range of applicat…
The bootstrap procedure has emerged as a general framework to construct prediction intervals for future observations in autoregressive time series models. Such models with outlying data points are standard in real data applications, especially in the field of econometrics. These outlying data points tend to produce high forecast errors, which reduce the forecasting performances of the existing bootstrap prediction intervals calculated based on non-robust estimators. In the univariate and multivariate autoregressive time series, we propose a robust bootstrap algorithm for constructing predictio…
The gmwm R package for inference on time series models is mainly based on the quantity called wavelet variance which is derived from a wavelet decomposition of a time series. This quantity provides a means to summarize and graphically represent the features of time series in order to identify possible models. Moreover, it is used as a moment condition for model estimation through the generalized method of wavelet moments. Based on the latter method, this package not only provides an alternative method to estimate classical ARMA models but also delivers a general framework for the robust estima…
This work proposes a novel method to robustly and accurately model time series with heavy-tailed noise, in non-stationary scenarios. In many practical application time series have heavy-tailed noise that significantly impacts the performance of classical forecasting models; in particular, accurately modeling a distribution over extreme events is crucial to performing accurate time series anomaly detection. We propose a Spliced Binned-Pareto distribution which is both robust to extreme observations and allows accurate modeling of the full distribution. Our method allows the capture of time depe…
This paper studies the problem of robustly learning the correlation function for a univariate time series with the presence of noise, outliers and missing entries. The outliers or anomalies considered here are sparse and rare events that deviate from normality which is depicted by a correlation function and an uncertainty condition. This general formulation is applied to univariate time series of event counts (or non-negative time series) where the correlation is a log-linear function with the uncertainty condition following the Poisson distribution. Approximations to the sparsity constraint, …
Robust optimal reconciliation for hierarchical time series forecasting with M-estimation
School of Economics and Management, Beihang University（北京航空航天大学经济管理学院）；Data Management Department, Industrial and Commercial Bank of China Limited（中国工商银行股份有限公司数据管理部）；MOE Key Laboratory of Complex System Analysis and Management Decision, Beihang University（北京航空航天大学复杂系统分析与管理决策教育部重点实验室）；Department of Mathematics, The University of Manchester（曼彻斯特大学数学系）
Quantmetry（Quantmetry公司）；ENSIIE（法国国立高等工业工程学校）；Université Paris Saclay（巴黎萨克雷大学）
We propose a robust principal component analysis (RPCA) framework to recover low-rank and sparse matrices from temporal observations. We develop an online version of the batch temporal algorithm in order to process larger datasets or streaming data. We empirically compare the proposed approaches with different RPCA frameworks and show their effectiveness in practical situations.
The stability and persistence of web services are important to Internet companies to improve user experience and business performances. To keep eyes on numerous metrics and report abnormal situations, time series anomaly detection methods are developed and applied by various departments in companies and institutions. In this paper, we proposed a robust anomaly detection algorithm (MEDIFF) to monitor online business metrics in real time. Specifically, a decomposition method using robust statistical metric--median--of the time series was applied to decouple the trend and seasonal components. Wit…
In this paper, we propose a robust data-driven process model whose hyperparameters are robustly estimated using the Schweppe-type generalized maximum likelihood estimator. The proposed model is trained on recorded time-series data of voltage phasors and power injections to perform a time-series stochastic power flow calculation. Power system data are often corrupted with outliers caused by large errors, fault conditions, power outages, and extreme weather, to name a few. The proposed model downweights vertical outliers and bad leverage points in the measurements of the training dataset. The we…
This paper presents a novel, closed-form, and data/computation efficient online anomaly detection algorithm for time-series data. The proposed method, dubbed RPE, is a window-based method and in sharp contrast to the existing window-based methods, it is robust to the presence of anomalies in its window and it can distinguish the anomalies in time-stamp level. RPE leverages the linear structure of the trajectory matrix of the time-series and employs a robust projection step which makes the algorithm able to handle the presence of multiple arbitrarily large anomalies in its window. A closed-form…
University of A Coruña（拉科鲁尼亚大学）；Sapienza University of Rome（罗马第一大学）；Technological Institute for Industrial Mathematics (ITMATI)（工业数学技术研究所）
Three robust methods for clustering multivariate time series from the point of view of generating processes are proposed. The procedures are robust versions of a fuzzy C-means model based on: (i) estimates of the quantile cross-spectral density and (ii) the classical principal component analysis. Robustness to the presence of outliers is achieved by using the so-called metric, noise and trimmed approaches. The metric approach incorporates in the objective function a distance measure aimed at neutralizing the effect of the outliers, the noise approach builds an artificial cluster expected to co…
Microsoft Research（微软研究院）；Australian National University（澳大利亚国立大学）；Indian Institute of Technology Kanpur（印度理工学院坎普尔分校）
We study the problem of robust time series analysis under the standard auto-regressive (AR) time series model in the presence of arbitrary outliers. We devise an efficient hard thresholding based algorithm which can obtain a consistent estimate of the optimal AR model despite a large fraction of the time series points being corrupted. Our algorithm alternately estimates the corrupted set of points and the model parameters, and is inspired by recent advances in robust regression and hard-thresholding methods. However, a direct application of existing techniques is hindered by a critical differe…
Exploring adversarial attack vectors and studying their effects on machine learning algorithms has been of interest to researchers. Deep neural networks working with time series data have received lesser interest compared to their image counterparts in this context. In a recent finding, it has been revealed that current state-of-the-art deep learning time series classifiers are vulnerable to adversarial attacks. In this paper, we introduce two local gradient based and one spectral density based time series data augmentation techniques. We show that a model trained with data obtained using our …
We propose a new autocorrelation measure for functional time series that we term spherical autocorrelation. It is based on measuring the average angle between lagged pairs of series after having been projected onto the unit sphere. This new measure enjoys several complimentary advantages compared to existing autocorrelation measures for functional data, since it both 1) describes a notion of sign or direction of serial dependence in the series, and 2) is more robust to outliers. The asymptotic properties of estimators of the spherical autocorrelation are established, and are used to construct …
Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift
College of Intelligence and Computing, Tianjin University, China（天津大学智能计算学院）；I2R, Agency for Science, Technology and Research, Singapore（新加坡科技研究局I2R）；Department of Computer Science, Khalifa University, UAE（阿联酋卡利法大学计算机科学系）
Adaptive Change Point Detection with Matrix Time Series: Leveraging Structured Mean Shifts
KLATASDS-MOE, School of Statistics, East China Normal University（华东师范大学统计学院）；Department of Statistics and Actuarial Science, University of Iowa（爱荷华大学统计与精算科学系）
