In this paper we study the asymptotic behaviour of empirical processes when parameters are estimated, assuming that the underlying sequence of random variables is long-range dependent. We show completely different phenomena compared to i.i.d. situation, as well as compared to ordinary empirical processes of long range dependent sequences. Applications include Kolmogorov-Smirnov and Cramer-Smirnov-von Mises goodness-of-fit statistics.
We consider the convergence of empirical processes indexed by functions that depend on an estimated parameter $η$ and give several alternative conditions under which the ``estimated parameter'' $η_n$ can be replaced by its natural limit $η_0$ uniformly in some other indexing set $Θ$. In particular we reconsider some examples treated by Ghoudi and Remillard [Asymptotic Methods in Probability and Statistics (1998) 171--197, Fields Inst. Commun. 44 (2004) 381--406]. We recast their examples in terms of empirical process theory, and provide an alternative general view which should be of wide appli…
The paper considers two main results. First one is the uniform bound for strong mixing coefficient of ARCH sequence. Second is the bound for maximum of residual empirical process in the same model. We illustrate their usefulness by proving robustness of two types of estimates (GM and minimum distance).
Optimal transport (OT) based data analysis is often faced with the issue that the underlying cost function is (partially) unknown. This paper is concerned with the derivation of distributional limits for the empirical OT value when the cost function and the measures are estimated from data. For statistical inference purposes, but also from the viewpoint of a stability analysis, understanding the fluctuation of such quantities is paramount. Our results find direct application in the problem of goodness-of-fit testing for group families, in machine learning applications where invariant transport…
We consider goodness-of-fit tests of symmetric stable distributions based on weighted integrals of the squared distance between the empirical characteristic function of the standardized data and the characteristic function of the standard symmetric stable distribution with the characteristic exponent $α$ estimated from the data. We treat $α$ as an unknown parameter, but for theoretical simplicity we also consider the case that $α$ is fixed. For estimation of parameters and the standardization of data we use maximum likelihood estimator (MLE) and an equivariant integrated squared error estimato…
We elaborate on a deconvolution method, used to estimate the empirical distribution of unknown parameters, as suggested recently by Efron (2013). It is applied to estimating the empirical distribution of the 'sampling probabilities' of m sampled items. The estimated empirical distribution is used to modify the Horvitz-Thompson estimator. The performance of the modified Horvitz-Thompson estimator is studied in two examples. In one example the sampling probabilities are estimated based on the number of visits until a response was obtained. The other example is based on real data from panel sampl…
This paper describes limiting behaviour of tail empirical process associated with long memory stochastic volatility models. We show that such process has dichotomous behaviour, according to an interplay between a Hurst parameter and a tail index. In particular, the limit may be non-Gaussian and/or degenerate, indicating an influence of long memory. On the other hand, tail empirical process with random levels never suffers from long memory. This is very desirable from a practical point of view, since such the process may be used to construct Hill estimator of the tail index. To prove our result…
By restricting the possible values of the proportion of null hypotheses that are true, the local false discovery rate (LFDR) can be estimated using as few as one comparison. The proportion of proteins with equivalent abundance was estimated to be about 20% for patient group I and about 90% for group II. The simultaneously-estimated LFDRs give approximately the same inferences as individual-protein confidence levels for group I but are much closer to individual-protein LFDR estimates for group II. Simulations confirm that confidence-based inference or LFDR-based inference performs markedly bett…
We consider the residual empirical process in random design regression with long memory errors. We establish its limiting behaviour, showing that its rates of convergence are different from the rates of convergence for to the empirical process based on (unobserved) errors. Also, we study a residual empirical process with estimated parameters. Its asymptotic distribution can be used to construct Kolmogorov-Smirnov, Cramér-Smirnov-von Mises, or other goodness-of-fit tests. Theoretical results are justified by simulation studies.
In this paper we investigate an indirect regression model characterized by the Radon transformation. This model is useful for recovery of medical images obtained by computed tomography scans. The indirect regression function is estimated using a series estimator motivated by a spectral cut-off technique. Further, we investigate the empirical process of residuals from this regression, and show that it satsifies a functional central limit theorem.
In the sparse sequence model, we consider a popular Bayesian multiple testing procedure and investigate for the first time its behaviour from the frequentist point of view. Given a spike-and-slab prior on the high-dimensional sparse unknown parameter, one can easily compute posterior probabilities of coming from the spike, which correspond to the well known local-fdr values, also called $\ell$-values. The spike-and-slab weight parameter is calibrated in an empirical Bayes fashion, using marginal maximum likelihood. The multiple testing procedure under study, called here the cumulative $\ell$-v…
We consider multivariate copula-based stationary time-series under Gaussian subordination. Observed time series are subordinated to long-range dependent Gaussian processes and characterized by arbitrary marginal copula distributions. First of all, we establish limit theorems for the marginal and quantile marginal empirical processes of multivariate stationary long-range dependent sequences under Gaussian subordination. Furthermore, we establish the asymptotic behavior of sequential empirical copula processes under non-restrictive smoothness assumptions. The limiting processes in the case of lo…
An empirical Bayes approach to the estimation of possibly sparse sequences observed in Gaussian white noise is set out and investigated. The prior considered is a mixture of an atom of probability at zero and a heavy-tailed density γ, with the mixing weight chosen by marginal maximum likelihood, in the hope of adapting between sparse and dense sequences. If estimation is then carried out using the posterior median, this is a random thresholding procedure. Other thresholding rules employing the same threshold can also be used. Probability bounds on the threshold chosen by the marginal maximum l…
Let $(X_k)_{k\geq1}$ be a Gaussian long-range dependent process with $EX_1=0$, $EX_1^2=1$ and covariance function $r(k)=k^{-D}L(k)$. For any measurable function $G$ let $(Y_k)_{k\geq1}=(G(X_k))_{k\geq1}$. We study the asymptotic behaviour of the associated sequential empirical process $\left(R_N(x,t)\right)$ with respect to a weighted norm $\|\cdot\|_w$. We show that, after an appropriate normalization, $\left(R_N(x,t)\right)$ converges weakly in the space of càdlàg functions with finite weighted norm to a Hermite process.
Given an arbitrary long but finite sequence of observations from a finite set, we construct a simple process that approximates the sequence, in the sense that with high probability the empirical frequency, as well as the empirical one-step transitions along a realization from the approximating process, are close to that of the given sequence. We generalize the result to the case where the one-step transitions are required to be in given polyhedra.
In this paper we propose two schemes for the recovery of the spectrum of a covariance matrix from the empirical covariance matrix, in the case where the dimension of the matrix is a subunitary multiple of the number of observations. We test, compare and analyze these on simulated data and also on some data coming from the stock market.
Let F be an unknown univariate distribution function to be estimated from a sample containing censored observations and tau be in dom(F). The author has derived a novel nonparametric estimator F_hat for F without making any assumptions regarding the nature of the censoring mechanism or the distribution function F. The distribution of F_hat(tau) can be easily and accurately estimated even for small sample sizes. The estimator F_hat has significantly outperformed the Kaplan Meier estimator in a simulation study with an exponential and a lognormal distribution functions F and a censoring mechanis…
In this paper, we propose a method to build a sequence of generalized empirical principal curves, with selected length, so that, in Hausdor distance, the images of the estimating principal curves converge in probability to the image of g.
We establish the Bahadur representation of sample quantiles for linear and some widely used nonlinear processes. Local fluctuations of empirical processes are discussed. Applications to the trimmed and Winsorized means are given. Our results extend previous ones by establishing sharper bounds under milder conditions and thus provide new insight into the theory of empirical processes for dependent random variables.
The Forward Search is an iterative algorithm for avoiding outliers in a regression analysis suggested by Hadi and Simonoff (J. Amer. Statist. Assoc. 88 (1993) 1264-1272), see also Atkinson and Riani (Robust Diagnostic Regression Analysis (2000) Springer). The algorithm constructs subsets of "good" observations so that the size of the subsets increases as the algorithm progresses. It results in a sequence of regression estimators and forward residuals. Outliers are detected by monitoring the sequence of forward residuals. We show that the sequences of regression estimators and forward residuals…
In this article, we construct empirical likelihood (EL)-weighted estimators of linear functionals of a probability measure in the presence of side information. Motivated by nuisance parameters in semiparametric models with possibly infinite dimensions, we consider the use of estimated constraint functions and allow the number of constraints to grow with the sample size. We study the asymptotic properties and efficiency gains. The results are used to construct improved estimators of parameters in structural equation models. The EL-weighted estimators of parameters are shown to have reduced vari…
We offer an umbrella type result which extends weak convergence of the classical empirical process on the line to that of more general processes indexed by functions of bounded variation. This extension is not contingent on the type of dependence of the underlying sequence of random variables. As a consequence we establish weak convergence for stationary empirical processes indexed by general classes of functions under alpha mixing conditions.
Empirical processes for stationary, causal sequences are considered. We establish empirical central limit theorems for classes of indicators of left half lines, absolutely continuous functions and piecewise differentiable functions. Sample path properties of empirical distribution functions are also discussed. The results are applied to linear processes and Markov chains.
We consider the estimation of a scalar parameter, when two estimators are available. The first is always consistent. The second is inconsistent in general, but has a smaller asymptotic variance than the first, and may be consistent if an assumption is satisfied. We propose to use the weighted sum of the two estimators with the lowest estimated mean-squared error (MSE). We show that this third estimator dominates the other two from a minimax-regret perspective: the maximum asymptotic-MSE-gain one may incur by using this estimator rather than one of the other estimators is larger than the maximu…
In this paper we propose a novel and practical variance reduction approach for additive functionals of dependent sequences. Our approach combines the use of control variates with the minimisation of an empirical variance estimate. We analyse finite sample properties of the proposed method and derive finite-time bounds of the excess asymptotic variance to zero. We apply our methodology to Stochastic Gradient MCMC (SGMCMC) methods for Bayesian inference on large data sets and combine it with existing variance reduction methods for SGMCMC. We present empirical results carried out on a number of b…
A reinforcement algorithm introduced by H.A. Simon \cite{Simon} produces a sequence of uniform random variables with memory as follows. At each step, with a fixed probability $p\in(0,1)$, $\hat U_{n+1}$ is sampled uniformly from $\hat U_1, \ldots, \hat U_n$, and with complementary probability $1-p$, $\hat U_{n+1}$ is a new independent uniform variable. The Glivenko-Cantelli theorem remains valid for the reinforced empirical measure, but not the Donsker theorem. Specifically, we show that the sequence of empirical processes converges in law to a Brownian bridge only up to a constant factor when…
In this article, we study critical points (zeros of derivative) of random polynomials. Take two deterministic sequences $\{a_n\}_{n\geq1}$ and $\{b_n\}_{n\geq1}$ of complex numbers whose limiting empirical measures are same. By choosing $ξ_n = a_n$ or $b_n$ with equal probability, define the sequence of polynomials by $P_n(z)=(z-ξ_1)\dots(z-ξ_n)$. We show that the limiting measure of zeros and critical points agree for this sequence of random polynomials under some assumption. We also prove a similar result for triangular array of numbers. A similar result for zeros of generalized derivative (…
In this note, we provide upper bounds on the expectation of the supremum of empirical processes indexed by Hölder classes of any smoothness and for any distribution supported on a bounded set in $\mathbb R^d$. These results can be alternatively seen as non-asymptotic risk bounds, when the unknown distribution is estimated by its empirical counterpart, based on $n$ independent observations, and the error of estimation is quantified by the integral probability metrics (IPM). In particular, the IPM indexed by a Hölder class is considered and the corresponding rates are derived. These results inte…
