We consider an integer-valued time series $Y=(Y_t)_{t\in\Z}$ where the models after a time $k^*$ is Poisson autoregressive with the conditional mean that depends on a parameter $θ^*\inΘ\subset\R^d$. The structure of the process before $k^*$ is unknown;? it could be any other integer-valued time series, that is, the process $Y$ could be nonstationary.? It is established that the maximum likelihood estimator of $θ^*$ computed on the nonstationary observations is consistent and asymptotically normal. Next, we carry out the sequential change-point detection in a large class of Poisson autoregressive models. We propose a monitoring scheme for detecting change in the model. The procedure is based on an updated estimator which is computed without the historical observations. The asymptotic behavior of the detector is studied, in particular, the above result on the inference in a nonstationary setting are applied to prove that the proposed procedure is consistent. A simulation study as well as a real data application are provided.
We propose NonStGM, a general nonparametric graphical modeling framework for studying dynamic associations among the components of a nonstationary multivariate time series. It builds on the framework of Gaussian Graphical Models (GGM) and stationary time series Gaussian Graphical model (StGM), and complements existing works on parametric graphical models based on change point vector autoregressions (VAR). Analogous to StGM, the proposed framework captures conditional noncorrelations (both intertemporal and contemporaneous) in the form of an undirected graph. In addition, to describe the more n…
We investigate the behavior of Fourier transforms for a wide class of nonstationary nonlinear processes. Asymptotic central and noncentral limit theorems are established for a class of nondegenerate and degenerate weighted $V$-statistics through the angle of Fourier analysis. The established theory for $V$-statistics provides a unified treatment for many important time and spectral domain problems in the analysis of nonstationary time series, ranging from nonparametric estimation to the inference of periodograms and spectral densities.
We review the advancement of nonstationary time series analysis from the perspective of Cowles Commission structural equation approach. We argue that despite the rich repertoire nonstationary time series analysis provides to analyze how do variables respond dynamically to shocks through the decomposition of a dynamic system into long-run and short-run relations, nonstationarity does not invalid the classical concerns of structural equation modeling -- identification and simultaneity bias. The same rank condition for identification holds for stationary and nonstationary data and some sort of in…
Gaussian couplings of partial sum processes are derived for the high-dimensional regime $d=o(n^{1/3})$. The coupling is derived for sums of independent random vectors and subsequently extended to nonstationary time series. Our inequalities depend explicitly on the dimension and on a measure of nonstationarity, and are thus also applicable to arrays of random vectors. To enable high-dimensional statistical inference, a feasible Gaussian approximation scheme is proposed. Applications to sequential testing and change-point detection are described.
The paper considers nonparametric specification tests of quantile curves for a general class of nonstationary processes. Using Bahadur representation and Gaussian approximation results for nonstationary time series, simultaneous confidence bands and integrated squared difference tests are proposed to test various parametric forms of the quantile curves with asymptotically correct type I error rates. A wild bootstrap procedure is implemented to alleviate the problem of slow convergence of the asymptotic results. In particular, our results can be used to test the trends of extremes of climate va…
We develop a new methodology for the fitting of nonstationary time series that exhibit nonlinearity, asymmetry, local persistence and changes in location scale and shape of the underlying distribution. In order to achieve this goal, we perform model selection in the class of piecewise stationary quantile autoregressive processes. The best model is defined in terms of minimizing a minimum description length criterion derived from an asymmetric Laplace likelihood. Its practical minimization is done with the use of genetic algorithms. If the data generating process follows indeed a piecewise quan…
This paper is concerned with the estimation of time-varying networks for high-dimensional nonstationary time series. Two types of dynamic behaviors are considered: structural breaks (i.e., abrupt change points) and smooth changes. To simultaneously handle these two types of time-varying features, a two-step approach is proposed: multiple change point locations are first identified based on comparing the difference between the localized averages on sample covariance matrices, and then graph supports are recovered based on a kernelized time-varying constrained $L_1$-minimization for inverse matr…
For multivariate stationary time series many important properties, such as partial correlation, graphical models and autoregressive representations are encoded in the inverse of its spectral density matrix. This is not true for nonstationary time series, where the pertinent information lies in the inverse infinite dimensional covariance matrix operator associated with the multivariate time series. This necessitates the study of the covariance of a multivariate nonstationary time series and its relationship to its inverse. We show that if the rows/columns of the infinite dimensional covariance …
This paper considers a class of nonparametric autoregressive models with nonstationarity. We propose a nonparametric kernel test for the conditional mean and then establish an asymptotic distribution of the proposed test. Both the setting and the results differ from earlier work on nonparametric autoregression with stationarity. In addition, we develop a new bootstrap simulation scheme for the selection of a suitable bandwidth parameter involved in the kernel test as well as the choice of a simulated critical value. The finite-sample performance of the proposed test is assessed using one simul…
We consider a time series model involving a fractional stochastic component, whose integration order can lie in the stationary/invertible or nonstationary regions and be unknown, and an additive deterministic component consisting of a generalized polynomial. The model can thus incorporate competing descriptions of trending behavior. The stationary input to the stochastic component has parametric autocorrelation, but innovation with distribution of unknown form. The model is thus semiparametric, and we develop estimates of the parametric component which are asymptotically normal and achieve an …
Tests for structural breaks in time series should ideally be sensitive to breaks in the parameter of interest, while being robust to nuisance changes. Statistical analysis thus needs to allow for some form of nonstationarity under the null hypothesis of no change. In this paper, estimators for integrated parameters of locally stationary time series are constructed and a corresponding functional central limit theorem is established, enabling change-point inference for a broad class of parameters under mild assumptions. The proposed framework covers all parameters which may be expressed as nonli…
We present online prediction methods for time series that let us explicitly handle nonstationary artifacts (e.g. trend and seasonality) present in most real time series. Specifically, we show that applying appropriate transformations to such time series before prediction can lead to improved theoretical and empirical prediction performance. Moreover, since these transformations are usually unknown, we employ the learning with experts setting to develop a fully online method (NonSTOP-NonSTationary Online Prediction) for predicting nonstationary time series. This framework allows for seasonality…
The goodness of the long-term prediction in the state-space model was evaluated using the squared long-term prediction error. In order to estimate the model parameters suitable for long-term prediction, we devised a modified log-likelihood corresponding to the long-term prediction error variance. Trend models and seasonally adjusted models with and without AR component are examined as examples.
We propose a new class of univariate nonstationary time series models, using the framework of modulated time series, which is appropriate for the analysis of rapidly-evolving time series as well as time series observations with missing data. We extend our techniques to a class of bivariate time series that are isotropic. Exact inference is often not computationally viable for time series analysis, and so we propose an estimation method based on the Whittle-likelihood, a commonly adopted pseudo-likelihood. Our inference procedure is shown to be consistent under standard assumptions, as well as …
Stationary subspace analysis (SSA) searches for linear combinations of the components of nonstationary vector time series that are stationary. These linear combinations and their number defne an associated stationary subspace and its dimension. SSA is studied here for zero mean nonstationary covariance processes. We characterize stationary subspaces and their dimensions in terms of eigenvalues and eigenvectors of certain symmetric matrices. This characterization is then used to derive formal statistical tests for estimating dimensions of stationary subspaces. Eigenstructure-based techniques ar…
We propose a novel Bayesian methodology for analyzing nonstationary time series that exhibit oscillatory behaviour. We approximate the time series using a piecewise oscillatory model with unknown periodicities, where our goal is to estimate the change-points while simultaneously identifying the potentially changing periodicities in the data. Our proposed methodology is based on a trans-dimensional Markov chain Monte Carlo (MCMC) algorithm that simultaneously updates the change-points and the periodicities relevant to any segment between them. We show that the proposed methodology successfully …
To capture the slowly time-varying spectral content of real-world time-series, a common paradigm is to partition the data into approximately stationary intervals and perform inference in the time-frequency domain. However, this approach lacks a corresponding nonstationary time-domain generative model for the entire data and thus, time-domain inference occurs in each interval separately. This results in distortion/discontinuity around interval boundaries and can consequently lead to erroneous inferences based on any quantities derived from the posterior, such as the phase. To address these shor…
Methods of estimation and forecasting for stationary models are well known in classical time series analysis. However, stationarity is an idealization which, in practice, can at best hold as an approximation, but for many time series may be an unrealistic assumption. We define a class of locally stationary processes which can lead to more accurate uncertainty quantification over making an invalid assumption of stationarity. This class of processes assumes the model parameters to be time-varying and parameterizes them in terms of a transformation of basis functions that ensures that the process…
Bilkent University（比尔肯特大学）；DataBoss A. S.（DataBoss股份公司）
We investigate nonlinear regression for nonstationary sequential data. In most real-life applications such as business domains including finance, retail, energy and economy, timeseries data exhibits nonstationarity due to the temporally varying dynamics of the underlying system. We introduce a novel recurrent neural network (RNN) architecture, which adaptively switches between internal regimes in a Markovian way to model the nonstationary nature of the given data. Our model, Markovian RNN employs a hidden Markov model (HMM) for regime transitions, where each regime controls hidden state transi…
Information from frequency bands in biomedical time series provides useful summaries of the observed signal. Many existing methods consider summaries of the time series obtained over a few well-known, pre-defined frequency bands of interest. However, these methods do not provide data-driven methods for identifying frequency bands that optimally summarize frequency-domain information in the time series. A new method to identify partition points in the frequency space of a multivariate locally stationary time series is proposed. These partition points signify changes across frequencies in the ti…
In this paper, we propose a fast, well-performing, and consistent method for segmenting a piecewise-stationary, linear time series with an unknown number of breakpoints. The time series model we use is the nonparametric Locally Stationary Wavelet model, in which a complete description of the piecewise-stationary second-order structure is provided by wavelet periodograms computed at multiple scales and locations. The initial stage of our method is a new binary segmentation procedure, with a theoretically justified and rapidly computable test criterion that detects breakpoints in wavelet periodo…
This article introduces a nonparametric approach to spectral analysis of a high-dimensional multivariate nonstationary time series. The procedure is based on a novel frequency-domain factor model that provides a flexible yet parsimonious representation of spectral matrices from a large number of simultaneously observed time series. Real and imaginary parts of the factor loading matrices are modeled independently using a prior that is formulated from the tensor product of penalized splines and multiplicative gamma process shrinkage priors, allowing for infinitely many factors with loadings incr…
Within the neurosciences, to observe variability across time in the dynamics of an underlying brain process is neither new nor unexpected. Wavelets are essential in analyzing brain signals because, even within a single trial, brain signals exhibit nonstationary behaviour. However, neurological signals generated within an experiment may also potentially exhibit evolution across trials (replicates). As neurologists consider localised spectra of brain signals to be most informative, here we develop a novel wavelet-based tool capable to formally represent process nonstationarities across both time…
While standard estimation assumes that all datapoints are from probability distribution of the same fixed parameters $θ$, we will focus on maximum likelihood (ML) adaptive estimation for nonstationary time series: separately estimating parameters $θ_T$ for each time $T$ based on the earlier values $(x_t)_{t<T}$ using (exponential) moving ML estimator $θ_T=\arg\max_θl_T$ for $l_T=\sum_{t<T} η^{T-t} \ln(ρ_θ(x_t))$ and some $η\in(0,1]$. Computational cost of such moving estimator is generally much higher as we need to optimize log-likelihood multiple times, however, in many cases it can be made i…
We explore the behavior of wind speed over time, using the Eastern Wind Dataset published by the National Renewable Energy Laboratory. This dataset gives wind speeds over three years at hundreds of potential wind farm sites. Wind speed analysis is necessary to the integration of wind energy into the power grid; short-term variability in wind speed affects decisions about usage of other power sources, so that the shape of the wind speed curve becomes as important as the overall level. To assess differences in intra-day time series, we propose a functional distance measure, the band distance, wh…
The Fourier Decomposition Method for nonlinear and nonstationary time series analysis
Indian Institute of Technology Delhi（德里印度理工学院）；Jaypee Institute of Information Technology（贾伊皮信息技术学院）；Samsung R & D Institute India - Delhi（三星印度研发中心-德里）
This article deals with detection of nonconstant long memory parameter in time series. The null hypothesis presumes stationary or nonstationary time series with constant long memory parameter, typically an I(d) series with d>-.5. The alternative corresponds to an increase in persistence and includes in particular an abrupt or gradual change from I(d_1) to I(d_2).
The state-space model and the Kalman filter provide us with unified and computationaly efficient procedure for computing the log-likelihood of the diverse type of time series models. This paper presents an algorithm for computing the gradient and the Hessian matrix of the log-likelihood by extending the Kalman filter without resorting to the numerical difference. Different from the previous paper(Kitagawa 2020), it is assumed that the observation noise variance R=1. It is known that for univariate time series, by maximizing the log-likelihood of this restricted model, we can obtain the same es…
Point forecasting of univariate time series is a challenging problem with extensive work having been conducted. However, nonparametric probabilistic forecasting of time series, such as in the form of quantiles or prediction intervals is an even more challenging problem. In an effort to expand the possible forecasting paradigms we devise and explore an extrapolation-based approach that has not been applied before for probabilistic forecasting. We present a novel quantile Fourier neural network is for nonparametric probabilistic forecasting of univariate time series. Multi-step predictions are p…
CREST, ENSAE, Université Paris Saclay（巴黎萨克雷大学国立统计与经济行政学院 CREST）；AGM UMR8088 University Paris-Seine（巴黎塞纳大学 AGM UMR8088）；CIMFAV, Universidad de Valparaiso（瓦尔帕莱索大学 CIMFAV）；CAM, Tianjin University（天津大学 CAM）
Exponential inequalities are main tools in machine learning theory. To prove exponential inequalities for non i.i.d random variables allows to extend many learning techniques to these variables. Indeed, much work has been done both on inequalities and learning theory for time series, in the past 15 years. However, for the non independent case, almost all the results concern stationary time series. This excludes many important applications: for example any series with a periodic behavior is non-stationary. In this paper, we extend the basic tools of Dedecker and Fan (2015) to nonstationary Mark…
This article introduces a nonparametric approach to multivariate time-varying power spectrum analysis. The procedure adaptively partitions a time series into an unknown number of approximately stationary segments, where some spectral components may remain unchanged across segments, allowing components to evolve differently over time. Local spectra within segments are fit through Whittle likelihood based penalized spline models of modified Cholesky components, which provide flexible nonparametric estimates that preserve positive definite structures of spectral matrices. The approach is formulat…
The oscillations observed in many time series, particularly in biomedicine, exhibit morphological variations over time. These morphological variations are caused by intrinsic or extrinsic changes to the state of the generating system, henceforth referred to as dynamics. To model these time series (including and specifically pathophysiological ones) and estimate the underlying dynamics, we provide a novel wave-shape oscillatory model. In this model, time-dependent variations in cycle shape occur along a manifold called the wave-shape manifold. To estimate the wave-shape manifold associated with…
The frequency-domain properties of nonstationary functional time series often contain valuable information. These properties are characterized through its time-varying power spectrum. Practitioners seeking low-dimensional summary measures of the power spectrum often partition frequencies into bands and create collapsed measures of power within bands. However, standard frequency bands have largely been developed through manual inspection of time series data and may not adequately summarize power spectra. In this article, we propose a framework for adaptive frequency band estimation of nonstatio…
Many studies of biomedical time series signals aim to measure the association between frequency-domain properties of time series and clinical and behavioral covariates. However, the time-varying dynamics of these associations are largely ignored due to a lack of methods that can assess the changing nature of the relationship through time. This article introduces a method for the simultaneous and automatic analysis of the association between the time-varying power spectrum and covariates. The procedure adaptively partitions the grid of time and covariate values into an unknown number of approxi…
In this article, using kernel convolution of order based dependent Dirichlet process (Griffin and Steel (2006)) we construct a nonstationary, nonseparable, nonparametric space-time process, which, as we show, satisfies desirable properties, and includes the stationary, separable, parametric processes as special cases. We also investigate the smoothness properties of our proposed model. Since our model entails an infinite random series, for Bayesian model fitting purpose we must either truncate the series or more appropriately consider a random number of summands, which renders the model dimens…
How does soil pollution affect a plant's circadian clock? Are there any differences between how the clock reacts when exposed to different concentrations of elements of the periodic table? If so, can we characterise these differences? We approach these questions by analysing and modelling circadian plant data, where the levels of expression of a luciferase reporter gene were measured at regular intervals over a number of days after exposure to different concentrations of lithium. A key aspect of circadian data analysis is to determine whether a time series (derived from experimental data) is `…
This work addresses the problem of missing data in time-series analysis focusing on (a) estimation of model parameters in the presence of missing data and (b) reconstruction of missing data. Standard approaches used to solve these problems like the maximum likelihood estimation or the Bayesian inference rely on a priori assumptions like the Gaussian or stationary behavior of missing data and might lead to biased results where these assumptions are unfulfilled. In order to go beyond, we extend the Finite Element Methodology (FEM) for Vector Auto-Regressive models with eXogenous factors and boun…
We consider estimation of quantile curves for a general class of nonstationary processes. Consistency and central limit results are obtained for local linear quantile estimates under a mild short-range dependence condition. Our results are applied to environmental data sets. In particular, our results can be used to address the problem of whether climate variability has changed, an important problem raised by IPCC (Intergovernmental Panel on Climate Change) in 2001.
This paper considers the inference of trends in multiple, nonstationary time series. To test whether trends are parallel to each other, we use a parallelism index based on the L2-distances between nonparametric trend estimators and their average. A central limit theorem is obtained for the test statistic and the test's consistency is established. We propose a simulation-based approximation to the distribution of the test statistic, which significantly improves upon the normal approximation. The test is also applied to devise a clustering algorithm. Finally, the finite-sample properties of the …
