We explore time-varying networks for high-dimensional locally stationary time series, using the large VAR model framework with both the transition and (error) precision matrices evolving smoothly over time. Two types of time-varying graphs are investigated: one containing directed edges of Granger causality linkages, and the other containing undirected edges of partial correlation linkages. Under the sparse structural assumption, we propose a penalised local linear method with time-varying weighted group LASSO to jointly estimate the transition matrices and identify their significant entries, and a time-varying CLIME method to estimate the precision matrices. The estimated transition and precision matrices are then used to determine the time-varying network structures. Under some mild conditions, we derive the theoretical properties of the proposed estimates including the consistency and oracle properties. In addition, we extend the methodology and theory to cover highly-correlated large-scale time series, for which the sparsity assumption becomes invalid and we allow for common factors before estimating the factor-adjusted time-varying networks. We provide extensive simulation studies and an empirical application to a large U.S. macroeconomic dataset to illustrate the finite-sample performance of our methods.
Stochastic networks are a plausible representation of the relational information among entities in dynamic systems such as living cells or social communities. While there is a rich literature in estimating a static or temporally invariant network from observation data, little has been done toward estimating time-varying networks from time series of entity attributes. In this paper we present two new machine learning methods for estimating time-varying networks, which both build on a temporally smoothed $l_1$-regularized logistic regression formalism that can be cast as a standard convex-optimi…
We consider the distribution of the turning point location of time series modeled as the sum of deterministic trend plus random noise. If the variables are modeled by shifted exponentials, whose location parameters define the trend, we provide a formula for computing the distribution of the turning point location and consequently to estimate a confidence interval for the location. We test this formula in simulated data series having a trend with asymmetric minimum, investigating the coverage rate as a function of a bandwidth parameter. The method is applied to estimate the confidence interval …
Functional time series have become an integral part of both functional data and time series analysis. Important contributions to methodology, theory and application for the prediction of future trajectories and the estimation of functional time series parameters have been made in the recent past. This paper continues this line of research by proposing a first principled approach to estimate invertible functional time series by fitting functional moving average processes. The idea is to estimate the coefficient operators in a functional linear filter. To do this a functional Innovations Algorit…
Examples of cyclic (periodic) behavior in geophysical data abound. In many cases the primary period is known, such as in daily measurements of rain, temperature, and sea level. However, many time series of measurements contain cycles of unknown or varying length. We consider the problem of estimating the unknown period in a time series.We review the basic methods, compare their performance through a simulation studyusing observed sea level data, apply them to an astronomical data set, and discuss generalizations of the methods.
Davis and Mikosch [7] introduced the extremogram as a flexible quantitative tool for measuring various types of extremal dependence in a stationary time series. There we showed some standard statistical properties of the sample extremogram. A major difficulty was the construction of credible confidence bands for the extremogram. In this paper, we employ the stationary bootstrap to overcome this problem. Moreover, we introduce the cross extremogram as a measure of extremal serial dependence between two or more time series. We also study the extremogram for return times between extremal events. …
Most time series observed in practice exhibit time-varying trend (first-order) and autocovariance (second-order) behaviour. Differencing is a commonly-used technique to remove the trend in such series, in order to estimate the time-varying second-order structure (of the differenced series). However, often we require inference on the second-order behaviour of the original series, for example, when performing trend estimation. In this article, we propose a method, using differencing, to jointly estimate the time-varying trend and second-order structure of a nonstationary time series, within the …
We propose a two-stage estimation method of variance components in time series models known as FDSLRMs, whose observations can be described by a linear mixed model (LMM). We based estimating variances, fundamental quantities in a time series forecasting approach called kriging, on the empirical (plug-in) best linear unbiased predictions of unobservable random components in FDSLRM. The method, providing invariant non-negative quadratic estimators, can be used for any absolutely continuous probability distribution of time series data. As a result of applying the convex optimization and the LMM m…
Signal-to-noise ratio (SNR) statistics play a central role in many applications. A common situation where SNR is studied is when a continuous time signal is sampled at a fixed frequency with some noise in the background. While estimation methods exist, little is known about its distribution when the noise is not weakly stationary. In this paper we develop a nonparametric method to estimate the distribution of an SNR statistic when the noise belongs to a fairly general class of stochastic processes that encompasses both short and long-range dependence, as well as nonlinearities. The method is b…
Time series of individual subjects have become a common data type in psychological research. These data allow one to estimate models of within-subject dynamics, and thereby avoid the notorious problem of making within-subjects inferences from between-subjects data, and naturally address heterogeneity between subjects. A popular model for these data is the Vector Autoregressive (VAR) model, in which each variable is predicted as a linear function of all variables at previous time points. A key assumption of this model is that its parameters are constant (or stationary) across time. However, in …
The estimation of time-varying networks for functional Magnetic Resonance Imaging (fMRI) data sets is of increasing importance and interest. In this work, we formulate the problem in a high-dimensional time series framework and introduce a data-driven method, namely Network Change Points Detection (NCPD), which detects change points in the network structure of a multivariate time series, with each component of the time series represented by a node in the network. NCPD is applied to various simulated data and a resting-state fMRI data set. This new methodology also allows us to identify common …
In this work we propose a new class of long-memory models with time-varying fractional parameter. In particular, the dynamics of the long-memory coefficient, $d$, is specified through a stochastic recurrence equation driven by the score of the predictive likelihood, as suggested by Creal et al. (2013) and Harvey (2013). We demonstrate the validity of the proposed model by a Monte Carlo experiment and an application to two real time series.
Measles is a significant source of global disease burden and child mortality. Measles vaccination through routine immunization (RI) programs in high-burden settings remains a challenge due to poor health care infrastructure and access. Supplementary immunization activities (SIA) in the form of vaccination campaigns are therefore implemented to prevent measles outbreaks by reducing the size of the susceptible population. The SIA efficacy, defined as the fraction of susceptible population immunized by an SIA, is a critical metric for assessing campaign effectiveness. We propose a discrete-time h…
Count-valued time series data are routinely collected in many application areas. We are particularly motivated to study the count time series of daily new cases, arising from COVID-19 spread. We propose two Bayesian models, a time-varying semiparametric AR(p) model for count and then a time-varying INGARCH model considering the rapid changes in the spread. We calculate posterior contraction rates of the proposed Bayesian methods with respect to average Hellinger metric. Our proposed structures of the models are amenable to Hamiltonian Monte Carlo (HMC) sampling for efficient computation. We su…
The Florence branch of an Italian supermarket chain recently implemented a strategy that permanently lowered the price of numerous store brands in several product categories. To quantify the impact of such a policy change, researchers often use synthetic control methods for estimating causal effects when a subset of units receive a single persistent treatment, and the rest are unaffected by the change. In our applications, however, competitor brands not assigned to treatment are likely impacted by the intervention because of substitution effects; more broadly, this type of interference occurs …
This paper studies methods for testing and estimating change-points in the covariance structure of a high-dimensional linear time series. The assumed framework allows for a large class of multivariate linear processes (including vector autoregressive moving average (VARMA) models) of growing dimension and spiked covariance models. The approach uses bilinear forms of the centered or non-centered sample variance-covariance matrix. Change-point testing and estimation are based on maximally selected weighted cumulated sum (CUSUM) statistics. Large sample approximations under a change-point regime …
We introduce a method for quantifying the inherent unpredictability of a continuous-valued time series via an extension of the differential Shannon entropy rate. Our extension, the specific entropy rate, quantifies the amount of predictive uncertainty associated with a specific state, rather than averaged over all states. We relate the specific entropy rate to popular `complexity' measures such as Approximate and Sample Entropies. We provide a data-driven approach for estimating the specific entropy rate of an observed time series. Finally, we consider three case studies of estimating specific…
The problem of change-point estimation is considered under a general framework where the data are generated by unknown stationary ergodic process distributions. In this context, the consistent estimation of the number of change-points is provably impossible. However, it is shown that a consistent clustering method may be used to estimate the number of change points, under the additional constraint that the correct number of process distributions that generate the data is provided. This additional parameter has a natural interpretation in many real-world applications. An algorithm is proposed t…
In this paper, we investigate temporal clusters of extremes defined as subsequent exceedances of high thresholds in a stationary time series. Two meaningful features of these clusters are the probability distribution of the cluster size and the ordinal patterns within a cluster. Since these patterns take only the ordinal structure of consecutive data points into account the method is robust under monotone transformations and measurement errors. We verify the existence of the corresponding limit distributions in the framework of regularly varying time series, develop non-parametric estimators a…
Cluster indices describe extremal behaviour of stationary time series. We consider their sliding blocks estimators. Using a modern theory of multivariate, regularly varying time series, we obtain central limit theorems under conditions that can be easily verified for a large class of models. In particular, we show that in the Peak over Threshold framework, sliding and disjoint blocks estimators have the same limiting variance.
In order to capture the dependence in the upper tail of a time series, we develop non-negative regularly-varying time series models that are constructed similarly to classical non-extreme ARMA models. Rather than fully characterizing tail dependence of the time series, we define the concept of weak tail stationarity which allows us to describe a regularly-varying time series through the tail pairwise dependence function (TPDF) which is a measure of pairwise extremal dependencies. We state consistency requirements among the finite-dimensional collections of the elements of a regularly-varying t…
We propose a new estimator to measure directed dependencies in time series. The dimensionality of data is first reduced using a new non-uniform embedding technique, where the variables are ranked according to a weighted sum of the amount of new information and improvement of the prediction accuracy provided by the variables. Then, using a greedy approach, the most informative subsets are selected in an iterative way. The algorithm terminates, when the highest ranked variable is not able to significantly improve the accuracy of the prediction as compared to that obtained using the existing sele…
A class of tests for change-point detection designed to be particularly sensitive to changes in the cross-sectional rank correlation of multivariate time series is proposed. The derived procedures are based on several multivariate extensions of Spearman's rho. Two approaches to carry out the tests are studied: the first one is based on resampling, the second one consists of estimating the asymptotic null distribution. The asymptotic validity of both techniques is proved under the null for strongly mixing observations. A procedure for estimating a key bandwidth parameter involved in both approa…
Cluster indices describe extremal behaviour of stationary time series. We consider runs estimators of cluster indices. Using a modern theory of multivariate, regularly varying time series, we obtain central limit theorems under conditions that can be easily verified for a large class of models. In particular, we show that blocks and runs estimators have the same limiting variance.
This paper proposes a flexible framework for inferring large-scale time-varying and time-lagged correlation networks from multivariate or high-dimensional non-stationary time series with piecewise smooth trends. Built on a novel and unified multiple-testing procedure of time-lagged cross-correlation functions with a fixed or diverging number of lags, our method can accurately disclose flexible time-varying network structures associated with complex functional structures at all time points. We broaden the applicability of our method to the structure breaks by developing difference-based nonpara…
Many real world systems exhibit cyclic behavior that is, for example, due to the nearly harmonic oscillations being perturbed by the strong fluctuations present in the regime of significant non-linearities. For the investigation of such sys- tems special techniques relaxing the assumption to periodicity are required. In this paper, we present the generalization of one of such techniques, namely the D2 phase dispersion statistic, to multidimensional datasets, especially suited for the analysis of the outputs from three-dimensional numerical simulations of the full magnetohydrodynamic equations.…
High-dimensional time series datasets are becoming increasingly common in many areas of biological and social sciences. Some important applications include gene regulatory network reconstruction using time course gene expression data, brain connectivity analysis from neuroimaging data, structural analysis of a large panel of macroeconomic indicators, and studying linkages among financial firms for more robust financial regulation. These applications have led to renewed interest in developing principled statistical methods and theory for estimating large time series models given only a relative…
We study the problem of estimating a temporally varying coefficient and varying structure (VCVS) graphical model underlying nonstationary time series data, such as social states of interacting individuals or microarray expression profiles of gene networks, as opposed to i.i.d. data from an invariant model widely considered in current literature of structural estimation. In particular, we consider the scenario in which the model evolves in a piece-wise constant fashion. We propose a procedure that minimizes the so-called TESLA loss (i.e., temporally smoothed L1 regularized regression), which al…
A multivariate, stationary time series is said to be jointly regularly varying if all its finite-dimensional distributions are multivariate regularly varying. This property is shown to be equivalent to weak convergence of the conditional distribution of the rescaled series given that, at a fixed time instant, its distance to the origin exceeds a threshold tending to infinity. The limit object, called the tail process, admits a decomposition in independent radial and angular components. Under an appropriate mixing condition, this tail process allows for a concise and explicit description of the…
In this article, we propose the fractional lower order covariance method (FLOC) for estimating the parameters of vector autoregressive process (VAR) of order $p$, $p\geq 1$ with symmetric stable noise. Further, we show the efficiency, accuracy, and simplicity of our methods through Monte-Carlo simulation.
Monash University（莫纳什大学）；University of California - Riverside（加州大学河滨分校）
Research into time series classification has tended to focus on the case of series of uniform length. However, it is common for real-world time series data to have unequal lengths. Differing time series lengths may arise from a number of fundamentally different mechanisms. In this work, we identify and evaluate two classes of such mechanisms -- variations in sampling rate relative to the relevant signal and variations between the start and end points of one time series relative to one another. We investigate how time series generated by each of these classes of mechanism are best addressed for…
Homogenization is an important and crucial step to improve the usage of observational data for climate analysis. This work is motivated by the analysis of long series of GNSS Integrated Water Vapour (IWV) data which have not yet been used in this context. This paper proposes a novel segmentation method that integrates a periodic bias and a heterogeneous, monthly varying, variance. The method consists in estimating first the variance using a robust estimator and then estimating the segmentation and periodic bias iteratively. This strategy allows for the use of the dynamic programming algorithm …
Understanding the functional architecture of the brain in terms of networks is becoming increasingly common. In most fMRI applications functional networks are assumed to be stationary, resulting in a single network estimated for the entire time course. However recent results suggest that the connectivity between brain regions is highly non-stationary even at rest. As a result, there is a need for new brain imaging methodologies that comprehensively account for the dynamic (i.e., non-stationary) nature of the fMRI data. In this work we propose the Smooth Incremental Graphical Lasso Estimation (…
In this paper we consider a regression model that allows for time series covariates as well as heteroscedasticity with a regression function that is modelled nonparametrically. We assume that the regression function changes at some unknown time $\lfloor ns_0\rfloor$, $s_0\in(0,1)$, and our aim is to estimate the (rescaled) change point $s_0$. The considered estimator is based on a Kolmogorov-Smirnov functional of the marked empirical process of residuals. We show consistency of the estimator and prove a rate of convergence of $O_P(n^{-1})$ which in this case is clearly optimal as there are onl…
We introduce a class of semiparametric time series models by assuming a quasi-likelihood approach driven by a latent factor process. More specifically, given the latent process, we only specify the conditional mean and variance of the time series and enjoy a quasi-likelihood function for estimating parameters related to the mean. This proposed methodology has three remarkable features: (i) no parametric form is assumed for the conditional distribution of the time series given the latent process; (ii) able for modelling non-negative, count, bounded/binary and real-valued time series; (iii) disp…
We consider the problem of estimating a signal from its warped observations. Such estimation is commonly performed by altering the observations through some inverse-warping, or solving a computationally demanding optimization formulation. While these may be unavoidable if observations are few, when large amounts of warped observations are available, the cost of running such algorithms can be prohibitive. We consider the scenario where we have many observations, and propose a computationally simple algorithm for estimating the function of interest. We demonstrate the utility of the algorithm on…
Bootstrap methods for estimating the long-run covariance of stationary functional time series are considered. We introduce a versatile bootstrap method that relies on functional principal component analysis, where principal component scores can be bootstrapped by maximum entropy. Two other bootstrap methods resample error functions, after the dependence structure being modeled linearly by a sieve method or nonlinearly by a functional kernel regression. Through a series of Monte-Carlo simulation, we evaluate and compare the finite-sample performances of these three bootstrap methods for estimat…
Signal data often contains missing values. Effective replacement (imputation) of the missing values can have significant positive effects on processing the signal. In this paper, we compare three commonly employed methods for estimating missing values in time series data: forward fill, backward fill, and mean fill. We carry out a large scale experimental analysis using 3,600 AR(1)-based simulated time series to determine the optimal method for estimating missing values. The results of the numerical experiments show that the forward and backward fill methods are better suited for times series w…
The predictability of a time series is determined by the sensitivity to initial conditions of its data generating process. In this paper our goal is to characterize this sensitivity from a finite sample by assuming few hypotheses on the data generating model structure. In order to measure the distance between two trajectories induced by a same noisy chaotic dynamic from two close initial conditions, a symmetric Kullback-Leiber divergence measure is used. Our approach allows to take into account the dependence of the residual variance on initial conditions. We show it is linked to a Fisher info…
University of California, Merced（加州大学默塞德分校）；Rakuten, Inc.（乐天株式会社）
While there has been a surge of recent interest in learning differential equation models from time series, methods in this area typically cannot cope with highly noisy data. We break this problem into two parts: (i) approximating the unknown vector field (or right-hand side) of the differential equation, and (ii) dealing with noise. To deal with (i), we describe a neural network architecture consisting of tensor products of one-dimensional neural shape functions. For (ii), we propose an alternating minimization scheme that switches between vector field training and filtering steps, together wi…
We adapt the classical definition of locally stationary processes in discrete-time to the continuous-time setting and obtain equivalent representations in the time and frequency domain. From this, a unique time-varying spectral density is derived using the Wigner-Ville spectrum. As an example, we investigate time-varying Lévy-driven state space processes, including the class of time-varying Lévy-driven CARMA processes. First, the connection between these two classes of processes is examined. Considering a sequence of time-varying Lévy-driven state space processes, we then give sufficient condi…
Estimating value-at-risk on time series data with possibly heteroscedastic dynamics is a highly challenging task. Typically, we face a small data problem in combination with a high degree of non-linearity, causing difficulties for both classical and machine-learning estimation algorithms. In this paper, we propose a novel value-at-risk estimator using a long short-term memory (LSTM) neural network and compare its performance to benchmark GARCH estimators. Our results indicate that even for a relatively short time series, the LSTM could be used to refine or monitor risk estimation processes and…
In this paper, we study nonparametric models allowing for locally stationary regressors and a regression function that changes smoothly over time. These models are a natural extension of time series models with time-varying coefficients. We introduce a kernel-based method to estimate the time-varying regression function and provide asymptotic theory for our estimates. Moreover, we show that the main conditions of the theory are satisfied for a large class of nonlinear autoregressive processes with a time-varying regression function. Finally, we examine structured models where the regression fu…
Carnegie Mellon University（卡内基梅隆大学）；Santa Fe Institute（圣塔菲研究所）
The literature on statistical learning for time series assumes the asymptotic independence or ``mixing' of the data-generating process. These mixing assumptions are never tested, nor are there methods for estimating mixing rates from data. We give an estimator for the $β$-mixing rate based on a single stationary sample path and show it is $L_1$-risk consistent.
Dynamic conditional correlation (DCC) is a method that estimates the correlation between two time series across time. Although used primarily in finance so far, DCC has been proposed recently as a model-based estimation method for quantifying functional connectivity during fMRI experiments. DCC could also be used to estimate the dynamic correlation between other types of time series such as local field potentials (LFP's) or spike trains recorded from distinct brain areas. DCC has very nice properties compared to other existing methods, but its applications for neuroscience are currently limite…
VISTA-SSM: Varying and Irregular Sampling Time-series Analysis via State Space Models
Johns Hopkins University（约翰斯·霍普金斯大学）；Institute for Psycholinguistics and Digital Health, United States of America（美国心理语言学和健康数字研究所）；New York University School of Medicine（纽约大学医学院）；University of Houston（休斯顿大学）
Uncertain differential equations have a wide range of applications. How to obtain estimated values of unknown parameters in uncertain differential equations through observations has always been a subject of concern and research, many methods have been developed to estimate unknown parameters. However, these parameters are constants. In this paper, the method of least squares estimation is recast for estimating the unknown time-varying parameters in uncertain differential equations. A set of unknown time-varying parameter estimates will be obtained, and then the unknown time-varying parameters …
This article develops new closed-form variance expressions for power analyses for commonly used difference-in-differences (DID) and comparative interrupted time series (CITS) panel data estimators. The main contribution is to incorporate variation in treatment timing into the analysis. The power formulas also account for other key design features that arise in practice: autocorrelated errors, unequal measurement intervals, and clustering due to the unit of treatment assignment. We consider power formulas for both cross-sectional and longitudinal models and allow for covariates. An illustrative…
We discuss the recent paper on "excursion effect" by T. Qian et al. (2020). We show that the methods presented have close relationships to others in the literature, in particular to a series of papers by Robins, Hernán and collaborators on analyzing observational studies as a series of randomized trials. There is also a close relationship to the history-restricted and the history-adjusted marginal structural models (MSM). Important differences and their methodological implications are clarified. We also demonstrate that the excursion effect can depend on the design and discuss its suitability …
Estimating the individual treatment effect (ITE) from observational data is meaningful and practical in healthcare. Existing work mainly relies on the strong ignorability assumption that no hidden confounders exist, which may lead to bias in estimating causal effects. Some studies consider the hidden confounders are designed for static environment and not easily adaptable to a dynamic setting. In fact, most observational data (e.g., electronic medical records) is naturally dynamic and consists of sequential information. In this paper, we propose Deep Sequential Weighting (DSW) for estimating I…
Estimating dynamic treatment regimes (DTRs) from retrospective observational data is challenging as some degree of unmeasured confounding is often expected. In this work, we develop a framework of estimating properly defined "optimal" DTRs with a time-varying instrumental variable (IV) when unmeasured covariates confound the treatment and outcome, rendering the potential outcome distributions only partially identified. We derive a novel Bellman equation under partial identification, use it to define a generic class of estimands (termed IV-optimal DTRs), and study the associated estimation prob…
Researchers collecting intensive longitudinal data (ILD) are increasingly looking to model psychological processes, such as emotional dynamics, that organize and adapt across time in complex and meaningful ways. This is also the case for researchers looking to characterize the impact of an intervention on individual behavior. To be useful, statistical models must be capable of characterizing these processes as complex, time-dependent phenomenon, otherwise only a fraction of the system dynamics will be recovered. In this paper we introduce a Square-Root Second-Order Extended Kalman Filtering ap…
