Adaptive estimation of linear functionals over a collection of parameter spaces is considered. A between-class modulus of continuity, a geometric quantity, is shown to be instrumental in characterizing the degree of adaptability over two parameter spaces in the same way that the usual modulus of continuity captures the minimax difficulty of estimation over a single parameter space. A general construction of optimally adaptive estimators based on an ordered modulus of continuity is given. The results are complemented by several illustrative examples.
This is a technical appendix to "Adaptive estimation of stationary Gaussian fields". We present several proofs that have been skipped in the main paper.
An adaptive nonparametric estimation procedure is constructed for the estimation problem of heteroscedastic regression when the noise variance depends on the unknown regression. A non-asymptotic upper bound for a quadratic risk (an oracle inequality) is constructed.
We focus on the nonparametric density estimation problem with directional data. We propose a new rule for bandwidth selection for kernel density estimation. Our procedure is automatic, fully data-driven and adaptive to the smoothness degree of the density. We obtain an oracle inequality and optimal rates of convergence for the L2 error. Our theoretical results are illustrated with simulations.
In this paper, we consider adaptive estimation of an unknown planar compact, convex set from noisy measurements of its support function on a uniform grid. Both the problem of estimating the support function at a point and that of estimating the convex set are studied. Data-driven adaptive estimators are proposed and their optimality properties are established. For pointwise estimation, it is shown that the estimator optimally adapts to every compact, convex set instead of a collection of large parameter spaces as in the conventional minimax theory of nonparametric estimation. For set estimatio…
We provide general adaptive upper bounds for estimating nonparametric functionals based on second order U-statistics arising from finite dimensional approximation of the infinite dimensional models. We then provide examples of functionals for which the theory produces rate optimally matching adaptive upper and lower bounds. Our results are automatically adaptive in both parametric and nonparametric regimes of estimation and are automatically adaptive and semiparametric efficient in the regime of parametric convergence rate.
The paper deals with asymptotic properties of the adaptive procedure proposed in the author paper (2007) for estimation of unknown nonparametric regression. We prove that this procedure is asymptotically efficient for a quadratic risk. It means that the asymptotic quadratic risk for this procedure coincides with a sharp lower bound.
Adaptive smoothness of function estimation in the three classical problems of the non-parametrical statistic in the three classical problems of the non-parametrical statistic
We study the adaptive minimax estimation of non-linear integral functionals of a density and extend the results obtained for linear and quadratic functionals to general functionals. The typical rate optimal non-adaptive minimax estimators of "smooth" non-linear functionals are higher order U-statistics. Since Lepski's method requires tight control of tails of such estimators, we bypass such calculations by a modification of Lepski's method which is applicable in such situations. As a necessary ingredient, we also provide a method to control higher order moments of minimax estimator of cubic in…
We address the problem of adaptive minimax estimation in white gaussian noise model under $L_p$--loss, $1\leq p\leq\infty,$ on the anisotropic Nikolskii classes. We present the estimation procedure based on a new data-driven selection scheme from the family of kernel estimators with varying bandwidths. For proposed estimator we establish so-called Lp-norm oracle inequality and use it for deriving minimax adaptive results. We prove the existence of rate-adaptive estimators and fully characterize behavior of the minimax risk for different relationships between regularity parameters and norm inde…
We consider the nonparametric robust estimation problem for regression models in continuous time with semi-Markov noises. An adaptive model selection procedure is proposed. Under general moment conditions on the noise distribution a sharp non-asymptotic oracle inequality for the robust risks is obtained and the robust efficiency is shown. It turns out that for semi-Markov models the robust minimax convergence rate may be faster or slower than the classical one.
For the sparse vector model, we consider estimation of the target vector, of its L2-norm and of the noise variance. We construct adaptive estimators and establish the optimal rates of adaptive estimation when adaptation is considered with respect to the triplet "noise level - noise distribution - sparsity". We consider classes of noise distributions with polynomially and exponentially decreasing tails as well as the case of Gaussian noise. The obtained rates turn out to be different from the minimax non-adaptive rates when the triplet is known. A crucial issue is the ignorance of the noise var…
In many practical settings one can sequentially and adaptively guide the collection of future data, based on information extracted from data collected previously. These sequential data collection procedures are known by different names, such as sequential experimental design, active learning or adaptive sensing/sampling. The intricate relation between data analysis and acquisition in adaptive sensing paradigms can be extremely powerful, and often allows for reliable signal estimation and detection in situations where non-adaptive sensing would fail dramatically. In this work we investigate the…
We build confidence balls for the common density $s$ of a real valued sample $X_1,...,X_n$. We use resampling methods to estimate the projection of $s$ onto finite dimensional linear spaces and a model selection procedure to choose an optimal approximation space. The covering property is ensured for all $n\geq2$ and the balls are adaptive over a collection of linear spaces.
The paper deals with asymptotic properties of the adaptive procedure proposed in the author paper, 2007, for estimating a unknown nonparametric regression. We prove that this procedure is asymptotically efficient for a quadratic risk, i.e. the asymptotic quadratic risk for this procedure coincides with the Pinsker constant which gives a sharp lower bound for the quadratic risk over all possible estimators.
Asymptotic efficiency is proved for the constructed in part 1 procedure, i.e. Pinsker's constant is found in the asymptotic lower bound for the minimax quadratic risk. It is shown that the asymptotic minimax quadratic risk of the constructed procedure coincides with this constant.
We consider the problem of estimation of a linear functional in the Gaussian sequence model where the unknown vector theta in R^d belongs to a class of s-sparse vectors with unknown s. We suggest an adaptive estimator achieving a non-asymptotic rate of convergence that differs from the minimax rate at most by a logarithmic factor. We also show that this optimal adaptive rate cannot be improved when s is unknown. Furthermore, we address the issue of simultaneous adaptation to s and to the variance sigma^2 of the noise. We suggest an estimator that achieves the optimal adaptive rate when both s …
We consider the problem of constructing Bayesian based confidence sets for linear functionals in the inverse Gaussian white noise model. We work with a scale of Gaussian priors indexed by a regularity hyper-parameter and apply the data-driven (slightly modified) marginal likelihood empirical Bayes method for the choice of this hyper-parameter. We show by theory and simulations that the credible sets constructed by this method have sub-optimal behaviour in general. However, by assuming "self-similarity" the credible sets have rate-adaptive size and optimal coverage. As an application of these r…
This paper is devoted to the estimation of the common marginal density function of weakly dependent processes. The accuracy of estimation is measured using pointwise risks. We propose a datadriven procedure using kernel rules. The bandwidth is selected using the approach of Goldenshluger and Lepski and we prove that the resulting estimator satisfies an oracle type inequality. The procedure is also proved to be adaptive (in a minimax framework) over a scale of Hölder balls for several types of dependence: stong mixing processes, $λ$-dependent processes or i.i.d. sequences can be considered usin…
In this paper a new estimator for the transition density $π$ of an homogeneous Markov chain is considered. We introduce an original contrast derived from regression framework and we use a model selection method to estimate $π$ under mild conditions. The resulting estimate is adaptive with an optimal rate of convergence over a large range of anisotropic Besov spaces $B_{2,\infty}^{(α_1,α_2)}$. Some simulations are also presented.
Gaussian mixture models are widely used to study clustering problems. These model-based clustering methods require an accurate estimation of the unknown data density by Gaussian mixtures. In Maugis and Michel (2009), a penalized maximum likelihood estimator is proposed for automatically selecting the number of mixture components. In the present paper, a collection of univariate densities whose logarithm is locally β-Hölder with moment and tail conditions are considered. We show that this penalized estimator is minimax adaptive to the β regularity of such densities in the Hellinger sense.
Adaptive estimation of a quadratic functional over both Besov and $L_p$ balls is considered. A collection of nonquadratic estimators are developed which have useful bias and variance properties over individual Besov and $L_p$ balls. An adaptive procedure is then constructed based on penalized maximization over this collection of nonquadratic estimators. This procedure is shown to be optimally rate adaptive over the entire range of Besov and $L_p$ balls in the sense that it attains certain constrained risk bounds.
This article deals with adaptive nonparametric estimation for Lévy processes observed at low frequency. For general linear functionals of the Lévy measure, we construct kernel estimators, provide upper risk bounds and derive rates of convergence under regularity assumptions. Our focus lies on the adaptive choice of the bandwidth, using model selection techniques. We face here a non-standard problem of model selection with unknown variance. A new approach towards this problem is proposed, which also allows a straightforward generalization to a classical density deconvolution framework.
We propose a method for estimating the entries of a large noisy matrix when the variance of the noise, $σ^2$, is unknown without putting any assumption on the rank of the matrix. We consider the estimator for $σ$ introduced by Gavish and Donoho \cite{Gavish} and give an upper bound on its mean squared error. Then with the estimate of the variance, we use a modified version of the Universal Singular Value Thresholding (USVT) algorithm introduced by Chatterjee \cite{Chatterjee} to estimate the noisy matrix. Finally, we give an upper bound on the mean squared error of the estimated matrix.
In this article, we consider a jump diffusion process (X_t)observed at discrete times t=0,Delta,...,nDelta. The sampling interval Delta tends to 0 and nDelta tends to infinity. We assume that (X_t) is ergodic, strictly stationary and exponentially β-mixing. We use a penalized least-square approach to compute two adaptive estimators of the drift function b. We provide bounds for the risks of the two estimators.
We address the problem of adaptive minimax density estimation on $\bR^d$ with $\bL_p$--loss on the anisotropic Nikol'skii classes. We fully characterize behavior of the minimax risk for different relationships between regularity parameters and norm indexes in definitions of the functional class and of the risk. In particular, we show that there are four different regimes with respect to the behavior of the minimax risk. We develop a single estimator which is (nearly) optimal in orderover the complete scale of the anisotropic Nikol'skii classes. Our estimation procedure is based on a data-drive…
The paper deals with asymptotic properties of the adaptive procedure proposed in the author paper, 2007, for estimating an unknown nonparametric regression. %\cite{GaPe1}. We prove that this procedure is asymptotically efficient for a quadratic risk, i.e. the asymptotic quadratic risk for this procedure coincides with the Pinsker constant which gives a sharp lower bound for the quadratic risk over all possible estimates
We provide in this paper a fully adaptive penalized procedure to select a covariance among a collection of models observing i.i.d replications of the process at fixed observation points. For this we generalize previous results of Bigot and al. and propose to use a data driven penalty to obtain an oracle inequality for the estimator. We prove that this method is an extension to the matricial regression model of the work by Baraud.
A $d$-dimensional nonparametric additive regression model with dependent observations is considered. Using the marginal integration technique and wavelets methodology, we develop a new adaptive estimator for a component of the additive regression function. Its asymptotic properties are investigated via the minimax approach under the $\mathbb{L}_2$ risk over Besov balls. We prove that it attains a sharp rate of convergence which turns to be the one obtained in the $\iid$ case for the standard univariate regression estimation problem.
We consider the nonparametric estimation problem of time-dependent multivariate functions observed in a presence of additive cylindrical Gaussian white noise of a small intensity. We derive minimax lower bounds for the $L^2$-risk in the proposed spatio-temporal model as the intensity goes to zero, when the underlying unknown response function is assumed to belong to a ball of appropriately constructed inhomogeneous time-dependent multivariate functions, motivated by practical applications. Furthermore, we propose both non-adaptive linear and adaptive non-linear wavelet estimators that are asym…
The partially observed linear Gaussian system of stochastic differential equations with low noise in observations is considered. A kernel-type estimators are used for estimation of the quadratic variation of the derivative of the limit of the observed process. Then this estimator is used for nonparametric estimation of the integral of the square of volatility of unobservable component. This estimator is also used for construction of substitution estimators in the case where the drift in observable component and the volatility of the state component depend on some unknown parameter. Then this s…
We study the estimation, in Lp-norm, of density functions defined on [0,1]^d. We construct a new family of kernel density estimators that do not suffer from the so-called boundary bias problem and we propose a data-driven procedure based on the Goldenshluger and Lepski approach that jointly selects a kernel and a bandwidth. We derive two estimators that satisfy oracle-type inequalities. They are also proved to be adaptive over a scale of anisotropic or isotropic Sobolev-Slobodetskii classes (which are particular cases of Besov or Sobolev classical classes). The main interest of the isotropic p…
Motivated by finance and technical applications, the objective of this paper is to consider adaptive estimation of regression and density distribution based on Fourier-Legendre expansion, and construction of confidence intervals - also adaptive. The estimators are asymptotically optimal and adaptive in the sense that they can adapt to unknown smoothness.
A two-class mixture model, where the density of one of the components is known, is considered. We address the issue of the nonparametric adaptive estimation of the unknown probability density of the second component. We propose a randomly weighted kernel estimator with a fully data-driven bandwidth selection method, in the spirit of the Goldenshluger and Lepski method. An oracle-type inequality for the pointwise quadratic risk is derived as well as convergence rates over Holder smoothness classes. The theoretical results are illustrated by numerical simulations.
This paper is concerned with adaptive kernel estimation of the Lévy density N(x) for bounded-variation pure-jump Lévy processes. The sample path is observed at n discrete instants in the "high frequency" context (Δ= Δ(n) tends to zero while nΔtends to infinity). We construct a collection of kernel estimators of the function g(x)=xN(x) and propose a method of local adaptive selection of the bandwidth. We provide an oracle inequality and a rate of convergence for the quadratic pointwise risk. This rate is proved to be the optimal minimax rate. We give examples and simulation results for processe…
In this article we consider the nonparametric robust estimation problem for regression models in continuous time with semi-Markov noises observed in discrete time moments. An adaptive model selection procedure is proposed. A sharp non-asymptotic oracle inequality for the robust risks is obtained. We obtain sufficient conditions on the frequency observations under which the robust efficiency is shown. It turns out that for the semi-Markov models the robust minimax convergence rate may be faster or slower than the classical one.
This work is concerned with the study of asymptotic properties of nonparametric density estimates in the framework of circular data. The estimation procedure here applied is based on wavelet thresholding methods: the wavelets used are the so-called Mexican needlets, which describe a nearly-tight frame on the circle. We study the asymptotic behaviour of the $L^{2}$-risk function for these estimates, in particular its adaptivity, proving that its rate of convergence is nearly optimal.
We show that under a linearity condition on the distribution of the predictors, the coefficient in single-index regression can be estimated with the same efficiency as in the case when the link function is known. Thus, the linearity condition seems to substitute for knowing the exact conditional distribution of the response given the linear combinations of the predictors.
We investigate the problem of deriving adaptive posterior rates of contraction on $\mathbb{L}^{\infty}$ balls in density estimation. Although it is known that log-density priors can achieve optimal rates when the true density is sufficiently smooth, adaptive rates were still to be proven. Here we establish that the so-called spike-and-slab prior can achieve adaptive and optimal posterior contraction rates. Along the way, we prove a generic $\mathbb{L}^{\infty}$ contraction result for log-density priors with independent wavelet coefficients. Interestingly, our approach is different from previou…
In the present paper, we derive lower bounds for the risk of the nonparametric empirical Bayes estimators. In order to attain the optimal convergence rate, we propose generalization of the linear empirical Bayes estimation method which takes advantage of the flexibility of the wavelet techniques. We present an empirical Bayes estimator as a wavelet series expansion and estimate coefficients by minimizing the prior risk of the estimator. As a result, estimation of wavelet coefficients requires solution of a well-posed low-dimensional sparse system of linear equations. The dimension of the syste…
We consider the recursive estimation of a regression functional where the explanatory variables take values in some functional space. We prove the almost sure convergence of such estimates for dependent functional data. Also we derive the mean quadratic error of the considered class of estimators. Our results are established with rates and asymptotic appear bounds, under strong mixing condition.
We introduce new quantile estimators with adaptive importance sampling. The adaptive estimators are based on weighted samples that are neither independent nor identically distributed. Using a new law of iterated logarithm for martingales, we prove the convergence of the adaptive quantile estimators for general distributions with nonunique quantiles thereby extending the work of Feldman and Tucker [Ann. Math. Statist. 37 (1996) 451--457]. We illustrate the algorithm with an example from credit portfolio risk analysis.
In this paper, we consider a functional linear regression model, where both the covariate and the response variable are functional random variables. We address the problem of optimal nonparametric estimation of the conditional expectation operator in this model. A collection of projection estimators over finite dimensional subspaces is first introduce. We provide a non-asymptotic bias-variance decomposition for the Mean Square Prediction error in the case where these subspaces are generated by the (empirical) PCA functional basis. The automatic trade-off is realized thanks to a model selection…
The paper deals with the non-parametric estimation in the regression with the multiplicative noise. Using the local polynomial fitting and the bayesian approach, we construct the minimax on isotropic Hölder class estimator. Next applying Lepski's method, we propose the estimator which is optimally adaptive over the collection of isotropic Hölder classes. To prove the optimality of the proposed procedure we establish, in particular, the exponential inequality for the deviation of locally bayesian estimator from the parameter to be estimated. These theoretical results are illustrated by simulati…
We derive concentration inequalities for maxima of empirical processes associated with Poisson point processes. The proofs are based on a careful application of Ledoux's entropy method. We demonstrate the utility of the obtained concentration inequalities by application to adaptive intensity estimation.
We consider a wavelet thresholding approach to adaptive variance function estimation in heteroscedastic nonparametric regression. A data-driven estimator is constructed by applying wavelet thresholding to the squared first-order differences of the observations. We show that the variance function estimator is nearly optimally adaptive to the smoothness of both the mean and variance functions. The estimator is shown to achieve the optimal adaptive rate of convergence under the pointwise squared error simultaneously over a range of smoothness classes. The estimator is also adaptively within a log…
We propose an estimation method that we call functional average variance estimation (FAVE), for estimating the EDR space in functional semiparametric regression model, based on kernel estimates of density and regression. Consistency results are then established for the estimator of the interest operator, and for the directions of EDR space. A simulation study that shows that the proposed approach performs as well as traditional ones is presented.
We consider the estimation of a density at a fixed point under a local differential privacy constraint, where the observations are anonymised before being available for statistical inference. We propose both a privatised version of a projection density estimator as well as a kernel density estimator and derive their minimax rates under a privacy constraint. There is a twofold deterioration of the minimax rates due to the anonymisation, which we show to be unavoidable by providing lower bounds. In both estimation procedures a tuning parameter has to be chosen. We suggest a variant of the classi…
We consider the problem of statistical learning for the intensity of a counting process with covariates. In this context, we introduce an empirical risk, and prove risk bounds for the corresponding empirical risk minimizers. Then, we give an oracle inequality for the popular algorithm of aggregation with exponential weights. This provides a way of constructing estimators that are adaptive to the smoothness and to the structure of the intensity. We prove that these estimators are adaptive over anisotropic Besov balls. The probabilistic tools are maximal inequalities using the generic chaining m…
We discuss parametric estimation of a degenerate diffusion system from time-discrete observations. The first component of the degenerate diffusion system has a parameter $θ_1$ in a non-degenerate diffusion coefficient and a parameter $θ_2$ in the drift term. The second component has a drift term parameterized by $θ_3$ and no diffusion term. Asymptotic normality is proved in three different situations for an adaptive estimator for $θ_3$ with some initial estimators for ($θ_1$ , $θ_2$), an adaptive one-step estimator for ($θ_1$ , $θ_2$ , $θ_3$) with some initial estimators for them, and a joint …
This paper is concerned with nonparametric estimation of the Lévy density of a pure jump Lévy process. The sample path is observed at $n$ discrete instants with fixed sampling interval. We construct a collection of estimators obtained by deconvolution methods and deduced from appropriate estimators of the characteristic function and its first derivative. We obtain a bound for the ${\mathbb L}^2$-risk, under general assumptions on the model. Then we propose a penalty function that allows to build an adaptive estimator. The risk bound for the adaptive estimator is obtained under additional assum…
We research adaptive maximum likelihood-type estimation for an ergodic diffusion process where the observation is contaminated by noise. This methodology leads to the asymptotic independence of the estimators for the variance of observation noise, the diffusion parameter and the drift one of the latent diffusion process. Moreover, it can lessen the computational burden compared to simultaneous maximum likelihood-type estimation. In addition to adaptive estimation, we propose a test to see if noise exists or not, and analyse real data as the example such that data contains observation noise wit…
Covariance function estimation is a fundamental task in multivariate functional data analysis and arises in many applications. In this paper, we consider estimating sparse covariance functions for high-dimensional functional data, where the number of random functions p is comparable to, or even larger than the sample size n. Aided by the Hilbert--Schmidt norm of functions, we introduce a new class of functional thresholding operators that combine functional versions of thresholding and shrinkage, and propose the adaptive functional thresholding estimator by incorporating the variance effects o…
We build and study a data-driven procedure for the estimation of the stationary density f of an additive fractional SDE. To this end, we also prove some new concentrations bounds for discrete observations of such dynamics in stationary regime.
We consider adaptive estimation and statistical inference for high-dimensional graph-based linear models. In our model, the coordinates of regression coefficients correspond to an underlying undirected graph. Furthermore, the given graph governs the piecewise polynomial structure of the regression vector. In the adaptive estimation part, we apply graph-based regularization techniques and propose a family of locally adaptive estimators called the Graph-Piecewise-Polynomial-Lasso. We further study a one-step update of the Graph-Piecewise-Polynomial-Lasso for the problem of statistical inference.…
