We obtain an optimal bound for a Gaussian approximation of a large class of vector-valued random processes. Our results provide a substantial generalization of earlier results that assume independence and/or stationarity. Based on the decay rate of the functional dependence measure, we quantify the error bound of the Gaussian approximation using the sample size $n$ and the moment condition. Under the assumption of $p$th finite moment, with $p>2$, this can range from a worst case rate of $n^{1/2}$ to the best case rate of $n^{1/p}$.
Various nonparametric approaches for Bayesian spectral density estimation of stationary time series have been suggested in the literature, mostly based on the Whittle likelihood approximation. A generalization of this approximation has been proposed in Kirch et al. who prove posterior consistency for spectral density estimation in combination with the Bernstein-Dirichlet process prior for Gaussian time series. In this paper, we will extend the posterior consistency result to non-Gaussian time series by employing a general consistency theorem of Shalizi for dependent data and misspecified model…
We consider the problem of approximating sums of high-dimensional stationary time series by Gaussian vectors, using the framework of functional dependence measure. The validity of the Gaussian approximation depends on the sample size $n$, the dimension $p$, the moment condition and the dependence of the underlying processes. We also consider an estimator for long-run covariance matrices and study its convergence properties. Our results allow constructing simultaneous confidence intervals for mean vectors of high-dimensional time series with asymptotically correct coverage probabilities. A Gaus…
This chapter presents some novel information theoretic results for the analysis of stationary time series in the frequency domain. In particular, the spectral distribution that corresponds to the most uncertain or unpredictable time series with some values of the autocovariance function fixed, is the generalized von Mises spectral distribution. It is thus a maximum entropy spectral distribution and the corresponding stationary time series is called the generalized von Mises time series. The generalized von Mises distribution is used in directional statistics for modelling planar directions tha…
We consider a time series $X=\{X_k, k\in\mathbb{Z}\}$ with memory parameter $d\in\mathbb{R}$. This time series is either stationary or can be made stationary after differencing a finite number of times. We study the "Local Whittle Wavelet Estimator" of the memory parameter $d$. This is a wavelet-based semiparametric pseudo-likelihood maximum method estimator. The estimator may depend on a given finite range of scales or on a range which becomes infinite with the sample size. We show that the estimator is consistent and rate optimal if $X$ is a linear process and is asymptotically normal if $X$…
We tackle the issue of the blind prediction of a Gaussian time series. For this, we construct a projection operator build by plugging an empirical covariance estimation into a Schur complement decomposition of the projector. This operator is then used to compute the predictor. Rates of convergence of the estimates are given.
Gaussian couplings of partial sum processes are derived for the high-dimensional regime $d=o(n^{1/3})$. The coupling is derived for sums of independent random vectors and subsequently extended to nonstationary time series. Our inequalities depend explicitly on the dimension and on a measure of nonstationarity, and are thus also applicable to arrays of random vectors. To enable high-dimensional statistical inference, a feasible Gaussian approximation scheme is proposed. Applications to sequential testing and change-point detection are described.
We consider the problem of inferring the conditional independence graph (CIG) of a multivariate stationary dicrete-time Gaussian random process based on a finite length observation. Using information-theoretic methods, we derive a lower bound on the error probability of any learning scheme for the underlying process CIG. This bound, in turn, yields a minimum required sample-size which is necessary for any algorithm regardless of its computational complexity, to reliably select the true underlying CIG. Furthermore, by analysis of a simple selection scheme, we show that the information-theoretic…
We study a well-known estimator of the fractal index of a stochastic process. Our framework is very general and encompasses many models of interest; we show how to extend the theory of the estimator to a large class of non-Gaussian processes. Particular focus is on clarity and ease of implementation of the estimator and the associated asymptotic results, making it easy for practitioners to apply the methods. We additionally show how measurement noise in the observations will bias the estimator, potentially resulting in the practitioner erroneously finding evidence of fractal characteristics in…
Shiga University（滋贺大学）；Osaka Gas Co., Ltd.（大阪燃气株式会社）
In this paper, we consider the task of clustering a set of individual time series while modeling each cluster, that is, model-based time series clustering. The task requires a parametric model with sufficient flexibility to describe the dynamics in various time series. To address this problem, we propose a novel model-based time series clustering method with mixtures of linear Gaussian state space models, which have high flexibility. The proposed method uses a new expectation-maximization algorithm for the mixture model to estimate the model parameters, and determines the number of clusters us…
A semi-analytic method is proposed for the generation of realizations of a multivariate process of a given linear correlation structure and marginal distribution. This is an extension of a similar method for univariate processes, transforming the autocorrelation of the non-Gaussian process to that of a Gaussian process based on a piece-wise linear marginal transform from non-Gaussian to Gaussian marginal. The extension to multivariate processes involves the derivation of the autocorrelation matrix from the marginal transforms, which determines the generating vector autoregressive process. The …
This paper develops the theory and methods for modeling a stationary count time series via Gaussian transformations. The techniques use a latent Gaussian process and a distributional transformation to construct stationary series with very flexible correlation features that can have any pre-specified marginal distribution, including the classical Poisson, generalized Poisson, negative binomial, and binomial structures. Gaussian pseudo-likelihood and implied Yule-Walker estimation paradigms, based on the autocovariance function of the count series, are developed via a new Hermite expansion. Part…
Ergodicity breaking is a challenge for biological and psychological sciences. Ergodicity is a necessary condition for linear causal modeling. Long-range correlations and non-Gaussianity characterizing various biological and psychological measurements break ergodicity routinely, threatening our capacity for causal modeling. Long-range correlations (e.g., in fractional Gaussian noise, a.k.a. "pink noise") break ergodicity--in raw Gaussian series, as well as in some but not all standard descriptors of variability, i.e., in coefficient of variation (CV) and root mean square (RMS) but not standard …
We introduce a Gaussian process-based model for handling of non-stationarity. The warping is achieved non-parametrically, through imposing a prior on the relative change of distance between subsequent observation inputs. The model allows the use of general gradient optimization algorithms for training and incurs only a small computational overhead on training and prediction. The model finds its applications in forecasting in non-stationary time series with either gradually varying volatility, presence of change points, or a combination thereof. We evaluate the model on synthetic and real-world…
The ordinary spectrum is restricted in its applications, since it is based on the second order moments (auto and cross-covariances). Alternative approaches to spectrum analysis have been investigated based on other measures of dependence. One such approach was developed for univariate time series by the authors of this paper using the local Gaussian auto-spectrum based on the local Gaussian auto-correlations. This makes it possible to detect local structures in univariate time series that looks like white noise when investigated by the ordinary auto-spectrum. In this paper the local Gaussian a…
The purpose of this paper is to provide a discussion, with illustrating examples, on Bayesian forecasting for dynamic generalized linear models (DGLMs). Adopting approximate Bayesian analysis, based on conjugate forms and on Bayes linear estimation, we describe the theoretical framework and then we provide detailed examples of response distributions, including binomial, Poisson, negative binomial, geometric, normal, log-normal, gamma, exponential, Weibull, Pareto, beta, and inverse Gaussian. We give numerical illustrations for all distributions (except for the normal). Putting together all the…
When performing a time series analysis of continuous data, for example from climate or environmental problems, the assumption that the process is Gaussian is often violated. Therefore, we introduce two non-Gaussian autoregressive time series models that are able to fit skewed and heavy-tailed time series data. Our two models are based on the Tukey g-and-h transformation. We discuss parameter estimation, order selection, and forecasting procedures for our models and examine their performances in a simulation study. We demonstrate the usefulness of our models by applying them to two sets of wind…
This paper introduces and describes the R package ts.extend, which adds probability functions for stationary Gaussian ARMA models and some related utility functions for time-series. We show how to use the package to compute the density and distributions functions for models in this class, and generate random vectors from this model. The package allows the user to use marginal or conditional models using a simple syntax for conditioning variables and marginalised elements. This allows users to simulate time-series vectors from any stationary Gaussian ARMA model, even if some elements are condit…
Forecasting on sparse multivariate time series (MTS) aims to model the predictors of future values of time series given their incomplete past, which is important for many emerging applications. However, most existing methods process MTS's individually, and do not leverage the dynamic distributions underlying the MTS's, leading to sub-optimal results when the sparsity is high. To address this challenge, we propose a novel generative model, which tracks the transition of latent clusters, instead of isolated feature representations, to achieve robust modeling. It is characterized by a newly desig…
The problem of time series approximation by series of finite rank is considered from the viewpoint of signal extraction. For signal estimation, a weighted least-squares method is applied to the trajectory matrix of the considered time series. Matrix weights are chosen to obtain equal or approximately equal weights in the equivalent problem of time-series least-squares approximation. Several new methods are suggested and examined together with the Cadzow's iterative method. The questions of convergence, computational complexity, and accuracy are considered for the proposed methods. The methods …
A new stochastic model for daily precipitation occurrence processes observed at multiple locations is developed. The modeling concept is to use the indicator function and the elliptical shape of multivariate Gaussian distribution to represent the joint probabilities of daily precipitation occurrences. By using this concept, the number of parameters needed for precipitation occurrence modeling can be largely reduced when compared to the commonly used two-state Markov chain approach. With this parameter reduction, the modeling of spatio-temporal dependence of daily precipitation occurrence proce…
We study random series priors for estimating a functional parameter (f\in L^2[0,1]). We show that with a series prior with random truncation, Gaussian coefficients, and inverse gamma multiplicative scaling, it is possible to achieve posterior contraction at optimal rates and adaptation to arbitrary degrees of smoothness. We present general results that can be combined with existing rate of contraction results for various nonparametric estimation problems. We give concrete examples for signal estimation in white noise and drift estimation for a one-dimensional SDE.
We provide the strong approximation of empirical copula processes by a Gaussian process. In addition we establish a strong approximation of the smoothed empirical copula processes and a law of iterated logarithm.
We study posterior rates of contraction in Gaussian process regression with unbounded covariate domain. Our argument relies on developing a Gaussian approximation to the posterior of the leading coefficients of a Karhunen--Loéve expansion of the Gaussian process. The salient feature of our result is deriving such an approximation in the $L^2$ Wasserstein distance and relating the speed of the approximation to the posterior contraction rate using a coupling argument. Specific illustrations are provided for the Gaussian or squared-exponential covariance kernel.
The aim of this paper is first the detection of multiple abrupt changes of the long-range dependence (respectively self-similarity, local fractality) parameters from a sample of a Gaussian stationary times series (respectively time series, continuous-time process having stationary increments). The estimator of the $m$ change instants (the number $m$ is supposed to be known) is proved to satisfied a limit theorem with an explicit convergence rate. Moreover, a central limit theorem is established for an estimator of each long-range dependence (respectively self-similarity, local fractality) para…
For a given positive random variable $V>0$ and a given $Z\sim N(0,1)$ independent of $V$, we compute the scalar $t_0$ such that the distance between $Z\sqrt{V}$ and $Z\sqrt{t_0}$ in the $L^2(\R)$ sense, is minimal. We also consider the same problem in several dimensions when $V$ is a random positive definite matrix.
In this paper, we study modulus of continuity and rate of convergence of series of conditionally sub-Gaussian random fields. This framework includes both classical series representations of Gaussian fields and LePage series representations of stable fields. We enlighten their anisotropic properties by using an adapted quasi-metric instead of the classical Euclidean norm. We specify our assumptions in the case of shot noise series where arrival times of a Poisson process are involved. This allows us to state unified results for harmonizable (multi)operator scaling stable random fields through t…
We derive a Gaussian approximation result for the maximum of a sum of random vectors under $(2+ι)$-th moments. Our main theorem is abstract and nonasymptotic, and can be applied to a variety of statistical learning problems. The proof uses the Lindeberg telescopic sum device along with some other newly developed technical results.
In this paper, we present some asymptotic properties of the normalized inverse-Gaussian process. In particular, when the concentration parameter is large, we establish an analogue of the empirical functional central limit theorem, the strong law of large numbers and the Glivenko-Cantelli theorem for the normalized inverse-Gaussian process and its corresponding quantile process. We also derive a finite sum-representation that converges almost surely to the Ferguson and Klass representation of the normalized inverse-Gaussian process. This almost sure approximation can be used to simulate efficie…
In this work we consider regularized Wasserstein barycenters (average in Wasserstein distance) in Fourier basis. We prove that random Fourier parameters of the barycenter converge to some Gaussian random vector by distribution. The convergence rate has been derived in finite-sample case with explicit dependence on measures count ($n$) and the dimension of parameters ($p$).
This paper derives a new strong Gaussian approximation bound for the sum of independent random vectors. The approach relies on the optimal transport theory and yields \textit{explicit} dependence on the dimension size $p$ and the sample size $n$. This dependence establishes a new fundamental limit for all practical applications of statistical learning theory. Particularly, based on this bound, we prove approximation in distribution for the maximum norm in a high-dimensional setting ($p >n$).
A generalization of the classic Gaussian random variable to the family of Multi- Gaussian (MG) random variables characterized by shape parameter M > 0, in addition to the mean and the standard deviation, is introduced. The probability density function of the MG family members is the alternating series of the Gaussian functions with the suitably chosen heights and widths. In particular, for the integer values of M the series has finite number of terms and leads to flattened profiles, while reducing to classic Gaussian density for M = 1. For non-integer, positive values of M a convergent infinit…
In a general class of Bayesian nonparametric models, we prove that the posterior distribution can be asymptotically approximated by a Gaussian process. Our results apply to nonparametric exponential family that contains both Gaussian and non-Gaussian regression, and also hold for both efficient (root-n) and inefficient (non root-n) estimation. Our general approximation theorem does not rely on posterior conjugacy, and can be verified in a class of Gaussian process priors that has a smoothing spline interpretation [59, 44]. In particular, the limiting posterior measure becomes prior-free under …
In this paper, we aim to give a theoretical approximation for the penalty level of $\ell_{1}$-regularization problems. This can save much time in practice compared with the traditional methods, such as cross-validation. To achieve this goal, we develop two Gaussian approximation methods, which are based on a moderate deviation theorem and Stein's method respectively. Both of them give efficient approximations and have good performances in simulations. We apply the two Gaussian approximation methods into three types of ultra-high dimensional $\ell_{1}$ penalized regressions: lasso, square-root …
We investigate the asymptotic distribution of the maximum of a frequency smoothed estimate of the spectral coherence of a M-variate complex Gaussian time series with mutually independent components when the dimension M and the number of samples N both converge to infinity. If B denotes the smoothing span of the underlying smoothed periodogram estimator, a type I extreme value limiting distribution is obtained under the rate assumptions M N $\rightarrow$ 0 and M B $\rightarrow$ c $\in$ (0, +$\infty$). This result is then exploited to build a statistic with controlled asymptotic level for testin…
Context. Whenever correlation functions are used for inference about cosmological parameters in the context of a Bayesian analysis, the likelihood function of correlation functions needs to be known. Usually, it is approximated as a multivariate Gaussian, though this is not necessarily a good approximation. Aims. We show how to calculate a better approximation for the probability distribution of correlation functions, which we call "quasi-Gaussian". Methods. Using the exact univariate PDF as well as constraints on correlation functions previously derived, we transform the correlation functions…
It is well-known that non-linear approximation has an advantage over linear schemes in the sense that it provides comparable approximation rates to those of the linear schemes, but to a larger class of approximands. This was established for spline approximations and for wavelet approximations, and more recently by DeVore and Ron for homogeneous radial basis function (surface spline) approximations. However, no such results are known for the Gaussian function, the preferred kernel in machine learning and several engineering problems. We introduce and analyze in this paper a new algorithm for ap…
Near-Gaussian probability densities are common in many important physical applications. Here we develop an asymptotic expansion methodology for computing entropic functionals for such densities. The expansion proposed is a close relative of standard perturbation expansions in quantum field theory. We give novel results on the low-order effects of non-Gaussian even moments and asymmetry (e.g. skewness) on the entropy. The asymptotic expansion is also used to define a best fit maximum entropy density given a set of observed low order moments. The maximum entropy density estimation technique cons…
We derive a Gaussian approximation result for the maximum of a sum of high-dimensional random vectors. Specifically, we establish conditions under which the distribution of the maximum is approximated by that of the maximum of a sum of the Gaussian random vectors with the same covariance matrices as the original vectors. This result applies when the dimension of random vectors ($p$) is large compared to the sample size ($n$); in fact, $p$ can be much larger than $n$, without restricting correlations of the coordinates of these vectors. We also show that the distribution of the maximum of a sum…
