We study asymptotic properties of $M$-estimates of regression parameters in linear models in which errors are dependent. Weak and strong Bahadur representations of the $M$-estimates are derived and a central limit theorem is established. The results are applied to linear models with errors being short-range dependent linear processes, heavy-tailed linear processes and some widely used nonlinear time series.
This paper deals with the estimation of hidden periodicities in a non-linear regression model with stationary noise displaying cyclical dependence. Consistency and asymptotic normality are established for the least-squares estimates.
Quantile estimation in deconvolution problems is studied comprehensively. In particular, the more realistic setup of unknown error distributions is covered. Our plug-in method is based on a deconvolution density estimator and is minimax optimal under minimal and natural conditions. This closes an important gap in the literature. Optimal adaptive estimation is obtained by a data-driven bandwidth choice. As a side result, we obtain optimal rates for the plug-in estimation of distribution functions with unknown error distributions. The method is applied to a real data example.
We define a new bandwidth-dependent kernel density estimator that improves existing convergence rates for the bias, and preserves that of the variation, when the error is measured in $L_1$. No additional assumptions are imposed to the extant literature.
This manuscript is a supplemental document providing the omitted material for our paper entitled "Nonparametric kernel estimation of the probability density function of regression errors using estimated residuals" [arXiv:1010.0439]. The paper is submitted to Journal of Nonparametric Statistics.
This work studies the total variation regularized $\ell_2$ estimator (fused lasso) in the setting of a change point detection problem. Compared with existing works that focus on the sum of squared estimation errors, we give bound on the element-wise estimation error. Our bound is nearly optimal in the sense that the sum of squared error matches the best existing result, up to a logarithmic factor. This analysis of the element-wise estimation error allows a screening method that can approximately detect all the change points. We also generalize this method to the muitivariate setting, i.e., to …
We consider the nonparametric estimation problem of time-dependent multivariate functions observed in a presence of additive cylindrical Gaussian white noise of a small intensity. We derive minimax lower bounds for the $L^2$-risk in the proposed spatio-temporal model as the intensity goes to zero, when the underlying unknown response function is assumed to belong to a ball of appropriately constructed inhomogeneous time-dependent multivariate functions, motivated by practical applications. Furthermore, we propose both non-adaptive linear and adaptive non-linear wavelet estimators that are asym…
In this article we study the estimation of the location of jump points in the first derivative (referred to as kinks) of a regression function μin two random design models with different long-range dependent (LRD) structures. The method is based on the zero-crossing technique and makes use of high-order kernels. The rate of convergence of the estimator is contingent on the level of dependence and the smoothness of the regression function μ. In one of the models, the convergence rate is the same as the minimax rate for kink estimation in the fixed design scenario with i.i.d. errors which sugges…
The Rényi entropy is a generalization of the Shannon entropy and is widely used in mathematical statistics and applied sciences for quantifying the uncertainty in a probability distribution. We consider estimation of the quadratic Rényi entropy and related functionals for the marginal distribution of a stationary m-dependent sequence. The U-statistic estimators under study are based on the number of epsilon-close vector observations in the corresponding sample. A variety of asymptotic properties for these estimators are obtained (e.g., consistency, asymptotic normality, Poisson convergence). T…
This paper deals with the nonparametric density estimation of the regression error term assuming its independence with the covariate. The difference between the feasible estimator which uses the estimated residuals and the unfeasible one using the true residuals is studied. An optimal choice of the bandwidth used to estimate the residuals is given. We also study the asymptotic normality of the feasible kernel estimator and its rate-optimality.
In this paper we consider nonparametric estimation for dependent data, where the observations do not necessarily come from a linear process. We study density estimation and also discuss associated problems in nonparametric regression using the 2-mixing dependence measure. We compare the results under 2-mixing with those derived under the assumption that the process is linear. In the context of panel time series where one observes data from several individuals, it is often too strong to assume the joint linearity of processes. Instead the methods developed in this paper enable us to quantify th…
Consider the nonparametric regression model Y=m(X)+E, where the function m is smooth but unknown, and E is independent of X. An estimator of the density of the error term E is proposed and its weak consistency is obtained. The contribution of this paper is twofold. First, we evaluate the impact of the estimation of the regression function on the error density estimator. Secondly, the optimal choices of the first and second step bandwidths used for estimating the regression function and the error density are proposed. Further, we investigate the asymptotic normality of the error density estimat…
We study the Lp-integrated risk of some classical estimators of the density, when the observations are drawn from a strictly stationary sequence. The results apply to a large class of sequences, which can be non-mixing in the sense of Rosenblatt and long-range dependent. The main probabilistic tool is a new Rosenthal-type inequality for partial sums of BV functions of the variables. As an application, we give the rates of convergence of regular Histograms, when estimating the invariant density of a class of expanding maps of the unit interval with a neutral fixed point at zero. These Histogram…
This paper deals with the problem of estimating a slope parameter in a simple linear regression model, where independent variables have functional measurement errors. Measurement errors in independent variables, as is well known, cause biasedness of the ordinary least squares estimator. A general procedure for the bias reduction is presented in a finite sample situation, and some exact bias-reduced estimators are proposed. Also, it is shown that certain truncation procedures improve the mean square errors of the ordinary least squares and the bias-reduced estimators.
This paper considers maximum likelihood (ML) estimation in a large class of models with hidden Markov regimes. We investigate consistency of the ML estimator and local asymptotic normality for the models under general conditions which allow for autoregressive dynamics in the observable process, Markov regime sequences with covariate-dependent transition matrices, and possible model misspecification. A Monte Carlo study examines the finite-sample properties of the ML estimator in correctly specified and misspecified models. An empirical application is also discussed.
Random uniform sampling has been studied in various statistical tasks but few of them have covered the Q-error metric for cardinality estimation (CE). In this paper, we analyze the confidence intervals of random uniform sampling with and without replacement for single-table CE. Results indicate that the upper Q-error bound depends on the sample size and true cardinality. Our bound gives a rule-of-thumb for how large a sample should be kept for single-table CE.
We consider a general statistical estimation problem wherein binary labels across different observations are not independent conditioned on their feature vectors, but dependent, capturing settings where e.g. these observations are collected on a spatial domain, a temporal domain, or a social network, which induce dependencies. We model these dependencies in the language of Markov Random Fields and, importantly, allow these dependencies to be substantial, i.e do not assume that the Markov Random Field capturing these dependencies is in high temperature. As our main contribution we provide algor…
The problem of error density estimation for a functional single index model with dependent errors is studied. A Bayesian method is utilized to simultaneously estimate the bandwidths in the kernel-form error density and regression function, under an autoregressive error structure. For estimating both the regression function and error density, empirical studies show that the functional single index model gives improved estimation and prediction accuracies than any nonparametric functional regression considered. Furthermore, estimation of error density facilitates the construction of prediction i…
We study the nonparametric estimation of the branching rate $B(x)$ of a supercritical Bellman-Harris population: a particle with age $x$ has a random lifetime governed by $B(x)$; at its death time, it gives rise to $k \geq 2$ children with lifetimes governed by the same division rate and so on. We observe in continuous time the process over $[0,T]$. Asymptotics are taken as $T \rightarrow \infty$; the data are stochastically dependent and one has to face simultaneously censoring, bias selection and non-ancillarity of the number of observations. In this setting, under appropriate ergodicity pro…
For a class of parametric modal regression models with measurement error, a simulation extrapolation estimation procedure is proposed in this paper for estimating the modal regression coefficients. Large sample properties of the proposed estimation procedure, including the consistency and asymptotic normality, are thoroughly investigated. Simulation studies are conducted to evaluate its robustness to potential outliers and the effectiveness in reducing the bias caused by the measurement error.
Motivated by fluorescence lifetime measurements this paper considers the problem of nonparametric density estimation in the pile-up model. Adaptive nonparametric estimators are proposed for the pile-up model in its simple form as well as in the case of additional measurement errors. Furthermore, oracle type risk bounds for the mean integrated squared error (MISE) are provided. Finally, the estimation methods are assessed by a simulation study and the application to real fluorescence lifetime data.
Empirical relationships are derived for the expected sampling error of quantile estimations using Monte Carlo experiments for two frequency distributions frequently encountered in climate sciences. The relationships found are expressed as a scaling factor times the standard error of the mean; these give a quick tool to estimate the uncertainty of quantiles for a given finite sample size.
In this paper we study the problem of pointwise density estimation from observations with multiplicative measurement errors. We elucidate the main feature of this problem: the influence of the estimation point on the estimation accuracy. In particular, we show that, depending on whether this point is separated away from zero or not, there are two different regimes in terms of the rates of convergence of the minimax risk. In both regimes we develop kernel--type density estimators and prove upper bounds on their maximal risk over suitable nonparametric classes of densities. We show that the prop…
Maximum pseudo-likelihood (MPL) is a semiparametric estimation method often used to obtain the dependence parameters in copula models from data. It has been shown that despite being consistent, and in some cases efficient, MPL estimation can overestimate the level of dependence especially for small weakly dependent samples. We show that the MPL method uses the expected value of order statistics and we propose to use instead the median or the mode of the same order statistics. In a simulation study we compare the finite-sample performance of the proposed estimators with that of the original MPL…
Given a sample $\{X_i\}_{i=1}^n$ from $f_X$, we construct kernel density estimators for $f_Y$, the convolution of $f_X$ with a known error density $f_ε$. This problem is known as density estimation with Berkson error and has applications in epidemiology and astronomy. Little is understood about bandwidth selection for Berkson density estimation. We compare three approaches to selecting the bandwidth both asymptotically, using large sample approximations to the MISE, and at finite samples, using simulations. Our results highlight the relationship between the structure of the error $f_ε$ and the…
This paper proposes a novel method to estimate large panel data error-correction models with stationary/non-stationary covariates and spatially dependent errors, which allows for known/unknown group-specific patterns of slope heterogeneity. Analysis is based on composite quasi-likelihood (CQL) maximization which performs estimation and classification simultaneously. The proposed CQL estimator remains unbiased in the presence of misspecification of the unobserved individual/group-specific fixed effects; therefore, neither instrumental variables nor bias corrections/reductions are required. This…
Linear regression models are useful statistical tools to analyze data sets in several different fields. There are several methods to estimate the parameters of a linear regression model. These methods usually perform under normally distributed and uncorrelated errors with zero mean and constant variance. However, for some data sets error terms may not satisfy these or some of these assumptions. If error terms are correlated, such as the regression models with autoregressive (AR(p)) error terms, the Conditional Maximum Likelihood (CML) under normality assumption or the Least Square (LS) methods…
The Count-Min sketch is an important and well-studied data summarization method. It allows one to estimate the count of any item in a stream using a small, fixed size data sketch. However, the accuracy of the sketch depends on characteristics of the underlying data. This has led to a number of count estimation procedures which work well in one scenario but perform poorly in others. A practitioner is faced with two basic, unanswered questions. Which variant should be chosen when the data is unknown? Given an estimate, is its error sufficiently small to be trustworthy? We provide answers to thes…
When performing supervised learning with the model selected using validation error from sample splitting and cross validation, the minimum value of the validation error can be biased downward. We propose two simple methods that use the errors produced in the validating step to estimate the test error after model selection, and we focus on the situations where we select the model by minimizing the validation error and the randomized validation error. Our methods do not require model refitting, and the additional computational cost is negligible. In the setting of sample splitting, we show that,…
We study an unbiased estimator for the density of a sum of random variables that are simulated from a computer model. A numerical study on examples with copula dependence is conducted where the proposed estimator performs favourably in terms of variance compared to other unbiased estimators. We provide applications and extensions to the estimation of marginal densities in Bayesian statistics and to the estimation of the density of sums of random variables under Gaussian copula dependence.
The approximation of integral functionals with respect to a stationary Markov process by a Riemann-sum estimator is studied. Stationarity and the functional calculus of the infinitesimal generator of the process are used to get a better understanding of the estimation error and to prove a general error bound. The presented approach admits general integrands and gives a unifying explanation for different rates obtained in the literature. Several examples demonstrate how the general bound can be related to well-known function spaces.
We consider the recursive estimation of a regression functional where the explanatory variables take values in some functional space. We prove the almost sure convergence of such estimates for dependent functional data. Also we derive the mean quadratic error of the considered class of estimators. Our results are established with rates and asymptotic appear bounds, under strong mixing condition.
This paper is devoted to the estimation of the common marginal density function of weakly dependent processes. The accuracy of estimation is measured using pointwise risks. We propose a datadriven procedure using kernel rules. The bandwidth is selected using the approach of Goldenshluger and Lepski and we prove that the resulting estimator satisfies an oracle type inequality. The procedure is also proved to be adaptive (in a minimax framework) over a scale of Hölder balls for several types of dependence: stong mixing processes, $λ$-dependent processes or i.i.d. sequences can be considered usin…
Existing methods for the estimation of stable distribution parameters, such as those based on sample quantiles, sample characteristic functions or maximum likelihood generally assume an independent sample. Little attention has been paid to estimation from a dependent sample. In this paper, a method for the estimation of stable distribution parameters from a dependent sample is proposed based on the sample quantiles. The estimates are shown to be asymptotically normal. The asymptotic variance is calculated for stable moving average processes. Simulations from stable moving average processes are…
A $d$-dimensional nonparametric additive regression model with dependent observations is considered. Using the marginal integration technique and wavelets methodology, we develop a new adaptive estimator for a component of the additive regression function. Its asymptotic properties are investigated via the minimax approach under the $\mathbb{L}_2$ risk over Besov balls. We prove that it attains a sharp rate of convergence which turns to be the one obtained in the $\iid$ case for the standard univariate regression estimation problem.
The estimation of a probability p from repeated Bernoulli trials is considered in this paper. A sequential approach is followed, using a simple stopping rule. A closed-form expression and an upper bound are obtained for the mean absolute error of the unbiased estimator of p. The results given permit the estimation of an arbitrary probability with a prescribed level of normalized mean absolute error.
This article proposes a link between statistics and the theory of Dirichlet forms used to compute errors. The error calculus based on Dirichlet forms is an extension of classical Gauss' approach to error propagation. The aim of this paper is to derive error structures from measurements. The links with Fisher's information lay the foundations of a strong connection with experiment. We show that this connection behaves well towards changes of variables and is related to the theory of asymptotic statistics.
We propose in this work an original estimator of the conditional intensity of a marker-dependent counting process, that is, a counting process with covariates. We use model selection methods and provide a non asymptotic bound for the risk of our estimator on a compact set. We show that our estimator reaches automatically a convergence rate over a functional class with a given (unknown) anisotropic regularity. Then, we prove a lower bound which establishes that this rate is optimal. Lastly, we provide a short illustration of the way the estimator works in the context of conditional hazard estim…
We estimate density and regression functions for weak dependant datas. Using an exponential inequality obtained by Dedecker and Prieur and in a previous article of the author, we control the deviation between the estimator and the function itself. These results are applied to a large class of dynamical systems and lead to estimations of invariant densities and on the mapping itself.
In this paper we study covariance estimation with missing data. We consider missing data mechanisms that can be independent of the data, or have a time varying dependency. Additionally, observed variables may have arbitrary (non uniform) and dependent observation probabilities. For each mechanism, we construct an unbiased estimator and obtain bounds for the expected value of their estimation error in operator norm. Our bounds are equivalent, up to constant and logarithmic factors, to state of the art bounds for complete and uniform missing observations. Furthermore, for the more general non un…
This paper studies some temporal dependence properties and addresses the issue of parametric estimation for a class of state-dependent autoregressive models for nonlinear time series in which we assume a stochastic autoregressive coefficient depending on the first lagged value of the process itself. We call such a model state-dependent first-order autoregressive process, (SDAR). We introduce some assumptions under which this class of models is strictly stationary and uniformly ergodic and we establish consistency and asymptotic normality of the quasi-maximum likelihood estimator of the paramet…
This paper investigates the minimum mean square error (MMSE) estimation of x, given the observation y = Hx+n, when x and n are independent and Gaussian Mixture (GM) distributed. The introduction of GM distributions, represents a generalization of the more familiar and simpler Gaussian signal and Gaussian noise instance. We present the necessary theoretical foundation and derive the MMSE estimator for x in a closed form. Furthermore, we provide upper and lower bounds for its mean square error (MSE). These bounds are validated through Monte Carlo simulations.
Small area estimation has received enormous attention in recent years due to its wide range of application, particularly in policy making decisions. The variance based on direct sample size of small area estimator is unduly large and there is a need of constructing model based estimator with low mean squared prediction error (MSPE). Estimation of MSPE and in particular the bias correction of MSPE plays the central piece of small area estimation research. In this article, a new technique of bias correction for the estimated MSPE is proposed. It is shown that that the new MSPE estimator attains …
We study the non-parametric estimation of the value $θ(f )$ of a linear functional evaluated at an unknown density function f with support on $R_+$ based on an i.i.d. sample with multiplicative measurement errors. The proposed estimation procedure combines the estimation of the Mellin transform of the density $f$ and a regularisation of the inverse of the Mellin transform by a spectral cut-off. In order to bound the mean squared error we distinguish several scenarios characterised through different decays of the upcoming Mellin transforms and the smoothnes of the linear functional. In fact, we…
In this paper we have suggested difference-type estimator for estimation of population mean of the study variable y in the presence of measurement error using auxiliary information. The optimum estimator in the suggested estimator has been identified along with its mean square error formula. It has been shown that the suggested estimator performs more efficient then other existing estimators. An empirical study is also carried out to illustrate the merits of proposed method over other traditional methods.
In this paper, we built a new nonparametric regression estimator with the local linear method by using the mean squared relative error as a loss function when the data are subject to random right censoring. We establish the uniform almost sure consistency with rate over a compact set of the proposed estimator. Some simulations are given to show the asymptotic behavior of the estimate in different cases.
We tackle the issue of the blind prediction of a Gaussian time series. For this, we construct a projection operator build by plugging an empirical covariance estimation into a Schur complement decomposition of the projector. This operator is then used to compute the predictor. Rates of convergence of the estimates are given.
There has been considerable advance in understanding the properties of sparse regularization procedures in high-dimensional models. In time series context, it is mostly restricted to Gaussian autoregressions or mixing sequences. We study oracle properties of LASSO estimation of weakly sparse vector-autoregressive models with heavy tailed, weakly dependent innovations with virtually no assumption on the conditional heteroskedasticity. In contrast to current literature, our innovation process satisfy an $L^1$ mixingale type condition on the centered conditional covariance matrices. This conditio…
Gaussian processes that can be decomposed into a smooth mean function and a stationary autocorrelated noise process are considered and a fully automatic nonparametric method to simultaneous estimation of mean and auto-covariance functions of such processes is developed. Our empirical Bayes approach is data-driven, numerically efficient and allows for the construction of confidence sets for the mean function. Performance is demonstrated in simulations and real data analysis. The method is implemented in the R package eBsc that accompanies the paper.
We consider the estimation of large covariance and precision matrices from high-dimensional sub-Gaussian or heavier-tailed observations with slowly decaying temporal dependence. The temporal dependence is allowed to be long-range so with longer memory than those considered in the current literature. We show that several commonly used methods for independent observations can be applied to the temporally dependent data. In particular, the rates of convergence are obtained for the generalized thresholding estimation of covariance and correlation matrices, and for the constrained $\ell_1$ minimiza…
In this article, we consider the parameter estimation of regression model with pth order autoregressive (AR(p)) error term. We use the Maximum Lq-likelihood (MLq) estimation method that is proposed by Ferrari and Yang (2010a), as a robust alternative to the classical maximum likelihood (ML) estimation method to handle the outliers in the data. After exploring the MLq estimators for the parameters of interest, we provide some asymptotic properties of the resulting MLq estimators. We give a simulation study and a real data example to illustrate the performance of the new estimators over the ML e…
Measurements of a scalar linear Gauss-Markov process are sent over a fading channel. The fading channel is modeled as independent and identically distributed random variables with known realization at the receiver. The optimal estimator at the receiver is the Kalman filter. In contrast to the classical Kalman filter theory, given a random channel, the Kalman gain and the error covariance become random. Then the probability distribution function of expected estimation error and its outage probability can be chosen for estimation quality assessment. In this paper and in order to get the estimati…
