This manuscript makes two contributions to the field of change-point detection. In a generalchange-point setting, we provide a generic algorithm for aggregating local homogeneity testsinto an estimator of change-points in a time series. Interestingly, we establish that the errorrates of the collection of tests directly translate into detection properties of the change-pointestimator. This generic scheme is then applied to various problems including covariance change-point detection, nonparametric change-point detection and sparse multivariate mean change-point detection. For the latter, we derive minimax optimal rates that are adaptive to theunknown sparsity and to the distance between change-points when the noise is Gaussian. Forsub-Gaussian noise, we introduce a variant that is optimal in almost all sparsity regimes.
We consider the sequential change-point detection problem of detecting changes that are characterized by a subspace structure. Such changes are frequent in high-dimensional streaming data altering the form of the corresponding covariance matrix. In this work we present a Subspace-CUSUM procedure and demonstrate its first-order asymptotic optimality properties for the case where the subspace structure is unknown and needs to be simultaneously estimated. To achieve this goal we develop a suitable analytical methodology that includes a proper parameter optimization for the proposed detection sche…
Gaussian Graphical Model selection problem is considered. Concentration graph is identified by multiple decision procedure based on individual tests. Optimal unbiased individual tests are constructed. It is shown that optimal tests are equivalent to sample partial correlation tests. Associated multiple decision procedure is compared with standard procedure.
Sequential change-point detection for graphs is a fundamental problem for streaming network data types and has wide applications in social networks and power systems. Given fixed vertices and a sequence of random graphs, the objective is to detect the change-point where the underlying distribution of the random graph changes. In particular, we focus on the local change that only affects a subgraph. We adopt the classical Erdos-Renyi model and revisit the generalized likelihood ratio (GLR) detection procedure. The scan statistic is computed by sequentially estimating the most-likely subgraph wh…
Models with multiple change points are used in many fields; however, the theoretical properties of maximum likelihood estimators of such models have received relatively little attention. The goal of this paper is to establish the asymptotic properties of maximum likelihood estimators of the parameters of a multiple change-point model for a general class of models in which the form of the distribution can change from segment to segment and in which, possibly, there are parameters that are common to all segments. Consistency of the maximum likelihood estimators of the change points is establishe…
This paper reviews recent developments in fundamental limits and optimal algorithms for change point analysis. We focus on minimax optimal rates in change point detection and localisation, in both parametric and nonparametric models. We start with the univariate mean change point analysis problem and review the state-of-the-art results in the literature. We then move on to more complex data types and investigate general principles behind the optimal procedures that lead to minimax rate-optimal results.
Change-point detection methods are proposed for the case of temporary failures, or transient changes, when an unexpected disorder is ultimately followed by a readjustment and return to the initial state. A base distribution of the "in-control" state changes to an "out-of-control" distribution for unknown periods of time. Likelihood based sequential and retrospective tools are proposed for the detection and estimation of each pair of change-points. The accuracy of the obtained change-point estimates is assessed. Proposed methods offer simultaneous control the familywise false alarm and false re…
There are many different notions of optimality even in testing a single hypothesis. In the multiple testing area, the number of possibilities is very much greater. The paper first will describe multiplicity issues that arise in tests involving a single parameter, and will describe a new optimality result in that context. Although the example given is of minimal practical importance, it illustrates the crucial dependence of optimality on the precise specification of the testing problem. The paper then will discuss the types of expanded optimality criteria that are being considered when hypothes…
Given a times series ${\bf Y}$ in $\mathbb{R}^n$, with a piece-wise contant mean and independent components, the twin problems of change-point detection and change-point localization respectively amount to detecting the existence of times where the mean varies and estimating the positions of those change-points. In this work, we tightly characterize optimal rates for both problems and uncover the phase transition phenomenon from a global testing problem to a local estimation problem. Introducing a suitable definition of the energy of a change-point, we first establish in the single change-poin…
We study the problem of change point detection for covariance matrices in high dimensions. We assume that we observe a sequence {X_i}_{i=1,...,n} of independent and centered p-dimensional sub-Gaussian random vectors whose covariance matrices are piecewise constant. Our task is to recover with high accuracy the number and locations of the change points, which are assumed unknown. Our generic model setting allows for all the model parameters to change with n, including the dimension p, the minimal spacing between consecutive change points, the magnitude of smallest change size and the maximal Or…
A weighted Shiryaev-Roberts change detection procedure is shown to approximately minimize the expected delay to detection as well as higher moments of the detection delay among all change-point detection procedures with the given low maximal local probability of a false alarm within a window of a fixed length in pointwise and minimax settings for general non-i.i.d. data models and for the composite post-change hypothesis when the post-change parameter is unknown. We establish very general conditions for the models under which the weighted Shiryaev-Roberts procedure is asymptotically optimal. T…
In 1960s Shiryaev developed Bayesian theory of change detection in independent and identically distributed (i.i.d.) sequences. In Shiryaev's classical setting the goal is to minimize an average detection delay under the constraint imposed on the average probability of false alarm. Recently, Tartakovsky and Veeravalli (2005) developed a general Bayesian asymptotic change-point detection theory (in the classical setting) that is not limited to a restrictive i.i.d. assumption. It was proved that Shiryaev's detection procedure is asymptotically optimal under traditional average false alarm probabi…
We study the multivariate nonparametric change point detection problem, where the data are a sequence of independent $p$-dimensional random vectors whose distributions are piecewise-constant with Lipschitz densities changing at unknown times, called change points. We quantify the size of the distributional change at any change point with the supremum norm of the difference between the corresponding densities. We are concerned with the localization task of estimating the positions of the change points. In our analysis, we allow for the model parameters to vary with the total number of time poin…
We study change point detection and localization for univariate data in fully nonparametric settings in which, at each time point, we acquire an i.i.d. sample from an unknown distribution. We quantify the magnitude of the distributional changes at the change points using the Kolmogorov--Smirnov distance. We allow all the relevant parameters -- the minimal spacing between two consecutive change points, the minimal magnitude of the changes in the Kolmogorov--Smirnov distance, and the number of sample points collected at each time point -- to change with the length of time series. We generalize t…
The problem of univariate mean change point detection and localization based on a sequence of $n$ independent observations with piecewise constant means has been intensively studied for more than half century, and serves as a blueprint for change point problems in more complex settings. We provide a complete characterization of this classical problem in a general framework in which the upper bound $σ^2$ on the noise variance, the minimal spacing $Δ$ between two consecutive change points and the minimal magnitude $κ$ of the changes, are allowed to vary with $n$. We first show that consistent lo…
This paper considers the detection of change points in parallel data streams, a problem widely encountered when analyzing large-scale real-time streaming data. Each stream may have its own change point, at which its data has a distributional change. With sequentially observed data, a decision maker needs to declare whether changes have already occurred to the streams at each time point.Once a stream is declared to have changed, it is deactivated permanently so that its future data will no longer be collected. This is a compound decision problem in the sense that the decision maker may want to …
This work deals with a general problem of testing multiple hypotheses about the distribution of a discrete-time stochastic process. Both the Bayesian and the conditional settings are considered. The structure of optimal sequential tests is characterized.
Sequential change-point detection when the distribution parameters are unknown is a fundamental problem in statistics and machine learning. When the post-change parameters are unknown, we consider a set of detection procedures based on sequential likelihood ratios with non-anticipating estimators constructed using online convex optimization algorithms such as online mirror descent, which provides a more versatile approach to tackle complex situations where recursive maximum likelihood estimators cannot be found. When the underlying distributions belong to a exponential family and the estimator…
This paper studies optimal designs for linear regression models with correlated effects for single responses. We introduce the concept of rhombic design to reduce the computational complexity and find a semi-algebraic description for the D-optimality of a rhombic design via the Kiefer-Wolfowitz equivalence theorem. Subsequently, we show that the structure of an optimal rhombic design depends directly on the correlation structure of the random coefficients.
In this paper we explore the behaviour of dependent test statistics for testing of multiple hypothesis . To keep simplicity, we have considered a mixture normal model with equicorrelated correlation set up. With a simple linear transformation,the test statistics were decomposed into independent components, which, when conditioned appropriately generated independent variables. These were used to construct conditional tests , which were shown to be asymptotically optimal with power as large as that obtained using N.P.Lemma. We have pursued extensive simulation to support the claim.
Very long and noisy sequence data arise from biological sciences to social science including high throughput data in genomics and stock prices in econometrics. Often such data are collected in order to identify and understand shifts in trend, e.g., from a bull market to a bear market in finance or from a normal number of chromosome copies to an excessive number of chromosome copies in genetics. Thus, identifying multiple change points in a long, possibly very long, sequence is an important problem. In this article, we review both classical and new multiple change-point detection strategies. Co…
This paper investigates a novel offline change-point detection problem from an information-theoretic perspective. In contrast to most related works, we assume that the knowledge of the underlying pre- and post-change distributions are not known and can only be learned from the training sequences which are available. We further require the probability of the \emph{estimation error} to decay either exponentially or sub-exponentially fast (corresponding respectively to the large and moderate deviations regimes in information theory parlance). Based on the training sequences as well as the test se…
Theory and algorithms are developed for detecting changes in the distribution of statistically periodic random processes. The statistical periodicity is modeled using independent and periodically identically distributed processes, a new class of stochastic processes proposed by us. An algorithm is developed that is minimax asymptotically optimal as the false alarm rate goes to zero. Algorithms are also developed for the cases when the post-change distribution is not known or when there are multiple streams of observations. The modeling is inspired by real datasets encountered in cyber-physical…
Paired estimation of change in parameters of interest over a population plays a central role in several application domains including those in the social sciences, epidemiology, medicine and biology. In these domains, the size of the population under study is often very large, however, the number of observations available per individual in the population is very small (\emph{sparse observations}) which makes the problem challenging. Consider the setting with $N$ independent individuals, each with unknown parameters $(p_i, q_i)$ drawn from some unknown distribution on $[0, 1]^2$. We observe $X_…
We present novel retrospective change point detection approach based on optimal transport and geometric discrepancy. The method does not require any parametric assumptions about distributions separated by change points. It can be used both for single and multiple change point detection and estimation, while the number of change points is either known or unknown. This result is achieved by construction of a certain sliding window statistic from which change points can be derived with elementary convex geometry in a specific Hilbert space. The work is illustrated with computational examples, bot…
Optimal detection of general moment changes: Simultaneous mean and covariance change detection and beyond
University of Notre Dame（圣母大学）；University of California, San Diego（加利福尼亚大学圣地亚哥分校）；Auburn University（奥本大学）；Washington University in St. Louis（华盛顿大学圣路易斯分校）
We present a novel scheme to boost detection power for kernel maximum mean discrepancy based sequential change-point detection procedures. Our proposed scheme features an optimal sub-sampling of the history data before the detection procedure, in order to tackle the power loss incurred by the random sub-sample from the enormous history data. We apply our proposed scheme to both Scan $B$ and Kernel Cumulative Sum (CUSUM) procedures, and improved performance is observed from extensive numerical experiments.
ANEO；Centre Borelli - ENS Paris-saclay（巴黎萨克雷高等师范学院博雷利中心）；Criteo AI Lab（科韬人工智能实验室）；Crest, ENSAE（法国国立统计与经济管理学院 Crest 研究中心）
We consider the quickest change detection problem where both the parameters of pre- and post- change distributions are unknown, which prevents the use of classical simple hypothesis testing. Without additional assumptions, optimal solutions are not tractable as they rely on some minimax and robust variant of the objective. As a consequence, change points might be detected too late for practical applications (in economics, health care or maintenance for instance). Available constant complexity techniques typically solve a relaxed version of the problem, deeply relying on very specific probabili…
We study the problem of detecting a change in the mean of one-dimensional Gaussian process data. This problem is investigated in the setting of increasing domain (customarily employed in time series analysis) and in the setting of fixed domain (typically arising in spatial data analysis). We propose a detection method based on the generalized likelihood ratio test (GLRT), and show that our method achieves nearly asymptotically optimal rate in the minimax sense, in both settings. The salient feature of the proposed method is that it exploits in an efficient way the data dependence captured by t…
We propose a new multiple change-point detection framework for multivariate and non-Euclidean data. First, we combine graph-based statistics with wild binary segmentation or seeded binary segmentation to search for a pool of candidate change-points. We then prune the candidate change-points through a novel goodness-of-fit statistic. Numerical studies show that this new framework outperforms existing methods under a wide range of settings. The resulting change-points can further be arranged hierarchically based on the goodness-of-fit statistic. The new framework is illustrated on a Neuropixels …
This paper considers the use of recently proposed optimal transport-based multivariate test statistics, namely rank energy and its variant the soft rank energy derived from entropically regularized optimal transport, for the unsupervised nonparametric change point detection (CPD) problem. We show that the soft rank energy enjoys both fast rates of statistical convergence and robust continuity properties which lead to strong performance on real datasets. Our theoretical analyses remove the need for resampling and out-of-sample extensions previously required to obtain such rates. In contrast the…
In recent years, there has been an increasing demand on efficient algorithms for large scale change point detection problems. To this end, we propose seeded binary segmentation, an approach relying on a deterministic construction of background intervals, called seeded intervals, in which single change points are searched. The final selection of change points based on the candidates from seeded intervals can be done in various ways, adapted to the problem at hand. Thus, seeded binary segmentation is easy to adapt to a wide range of change point detection problems, let that be univariate, multiv…
The problem of quantifying uncertainty about the locations of multiple change points by means of confidence intervals is addressed. The asymptotic distribution of the change point estimators obtained as the local maximisers of moving sum statistics is derived, where the limit distributions differ depending on whether the corresponding size of changes is local, i.e. tends to zero as the sample size increases, or fixed. A bootstrap procedure for confidence interval generation is proposed which adapts to the unknown magnitude of changes and guarantees asymptotic validity both for local and fixed …
We present the group fused Lasso for detection of multiple change-points shared by a set of co-occurring one-dimensional signals. Change-points are detected by approximating the original signals with a constraint on the multidimensional total variation, leading to piecewise-constant approximations. Fast algorithms are proposed to solve the resulting optimization problems, either exactly or approximately. Conditions are given for consistency of both algorithms as the number of signals increases, and empirical evidence is provided to support the results on simulated and array comparative genomic…
We study the problem of change point localization in dynamic networks models. We assume that we observe a sequence of independent adjacency matrices of the same size, each corresponding to a realization of an unknown inhomogeneous Bernoulli model. The underlying distribution of the adjacency matrices are piecewise constant, and may change over a subset of the time points, called change points. We are concerned with recovering the unknown number and positions of the change points. In our model setting we allow for all the model parameters to change with the total number of time points, includin…
Change point analysis is a statistical tool to identify homogeneity within time series data. We propose a pruning approach for approximate nonparametric estimation of multiple change points. This general purpose change point detection procedure `cp3o' applies a pruning routine within a dynamic program to greatly reduce the search space and computational costs. Existing goodness-of-fit change point objectives can immediately be utilized within the framework. We further propose novel change point algorithms by applying cp3o to two popular nonparametric goodness of fit measures: `e-cp3o' uses E-s…
In this paper, we study the problem of multiple change-point detection for a univariate sequence under the epidemic setting, where the behavior of the sequence alternates between a common normal state and different epidemic states. This is a non-trivial generalization of the classical (single) epidemic change-point testing problem. To explicitly incorporate the alternating structure of the problem, we propose a novel model selection based approach for simultaneous inference on both change-points and alternating states. Using the same spirit as profile likelihood, we develop a two-stage alterna…
We propose a general framework to construct self-normalized multiple-change-point tests with time series data. The only building block is a user-specified one-change-point detecting statistic, which covers a wide class of popular methods, including cumulative sum process, outlier-robust rank statistics and order statistics. Neither robust and consistent estimation of nuisance parameters, selection of bandwidth parameters, nor pre-specification of the number of change points is required. The finite-sample performance shows that our proposal is size-accurate, robust against misspecification of t…
We consider the offline change point detection and localization problem in the context of piecewise stationary networks, where the observable is a finite sequence of networks. We develop algorithms involving some suitably modified CUSUM statistics based on adaptively trimmed adjacency matrices of the observed networks for both detection and localization of single or multiple change points present in the input data. We provide rigorous theoretical analysis and finite sample estimates evaluating the performance of the proposed methods when the input (finite sequence of networks) is generated fro…
The use of an Ornstein-Uhlenbeck (OU) process is ubiquitous in business, economics and finance to capture various price processes and evolution of economic indicators exhibiting mean-reverting properties. When structural changes happen, economic dynamics drastically change and the times at which these occur are of particular interest to policy makers, investors and financial product providers. This paper addresses the change-point problem under a generalised OU model and investigates the associated statistical inference. We propose two estimation methods to locate multiple change points and sh…
The extensive emergence of big data techniques has led to an increasing interest in the development of change-point detection algorithms that can perform well in a multivariate, possibly high-dimensional setting. In the current paper, we propose a new method for the consistent estimation of the number and location of multiple generalized change-points in multivariate, possibly high-dimensional, noisy data sequences. The number of change-points is allowed to increase with the sample size and the dimensionality of the given data sequence. Having a number of univariate signals, which constitute t…
We develop a new method for frequentist multiple testing with Bayesian prior information. Our procedure finds a new set of optimal p-value weights called the Bayes weights. Prior information is relevant to many multiple testing problems. Existing methods assume fixed, known effect sizes available from previous studies. However, the case of uncertain information is usually the norm. For a Gaussian prior on effect sizes, we show that finding the optimal weights is a non-convex problem. Despite the non-convexity, we give an efficient algorithm that solves this problem nearly exactly. We show that…
We present a new non-parametric statistic, called the weighed $\ell_2$ divergence, based on empirical distributions for sequential change detection. We start by constructing the weighed $\ell_2$ divergence as a fundamental building block for two-sample tests and change detection. The proposed statistic is proved to attain the optimal sample complexity in the offline setting. We then study the sequential change detection using the weighed $\ell_2$ divergence and characterize the fundamental performance metrics, including the average run length (ARL) and the expected detection delay (EDD). We al…
This paper considers the change-point problem for finite sequences of networks. To avoid the difficulty of computing the normalization coefficient, such as in Exponential random graphical models (ERGMs) and Markov networks, we construct a finite measure space with measure ratio statistics. A new performance measure of detection delay is proposed to detect the changes in distribution of the network. And an optimal sequential test is proposed under the performance measure. The good performance of the optimal sequential test is illustrated numerically on ERGMs and Erdos-Rényi network sequences.
We study nonparametric change-point estimation from indirect noisy observations. Focusing on the white noise convolution model, we consider two classes of functions that are smooth apart from the change-point. We establish lower bounds on the minimax risk in estimating the change-point and develop rate optimal estimation procedures. The results demonstrate that the best achievable rates of convergence are determined both by smoothness of the function away from the change-point and by the degree of ill-posedness of the convolution operator. Optimality is obtained by introducing a new technique …
In this paper we develop a generalized likelihood ratio scan method (GLRSM) for multiple change-points inference in piecewise stationary time series, which estimates the number and positions of change-points and provides a confidence interval for each change-point. The computational complexity of using GLRSM for multiple change-points detection is as low as $O(n(\log n)^3)$ for a series of length $n$. Consistency of the estimated numbers and positions of the change-points is established. Extensive simulation studies are provided to demonstrate the effectiveness of the proposed methodology unde…
The paper addresses a sequential changepoint detection problem for a general stochastic model, assuming that the observed data may be non-i.i.d. (i.e., dependent and non-identically distributed) and the prior distribution of the change point is arbitrary. Tartakovsky and Veeravalli (2005), Baron and Tartakovsky (2006), and, more recently, Tartakovsky (2017) developed a general asymptotic theory of changepoint detection for non-i.i.d.\ stochastic models, assuming the certain stability of the log-likelihood ratio process, in the case of simple hypotheses when both pre-change and post-change mode…
This article develops a method to construct the optimal sequential test for monitoring the changes in the distribution of finite observation sequences with a general dependence structure. This method allows us to prove that different optimal sequential tests can be constructed for different performance measures of detection delay times. We also provide a formula to calculate the value of the generalized out-of-control average run length for every optimal sequential test. Moreover, we show that there is an equivalent optimal control limit which does not depend on the test statistic directly whe…
Several variations of the Shiryaev-Roberts detection procedure in the context of the simple changepoint problem are considered: starting the procedure at $R_0=0$ (the original Shiryaev-Roberts procedure), at $R_0=r$ for fixed $r>0$, and at $R_0$ that has a quasi-stationary distribution. Comparisons of operating characteristics are made. The differences fade as the average run length to false alarm tends to infinity. It is shown that the Shiryaev-Roberts procedures that start either from a specially designed point $r$ or from the random "quasi-stationary" point are order-3 asymptotically optima…
