Adaptive estimation of a quadratic functional over both Besov and $L_p$ balls is considered. A collection of nonquadratic estimators are developed which have useful bias and variance properties over individual Besov and $L_p$ balls. An adaptive procedure is then constructed based on penalized maximization over this collection of nonquadratic estimators. This procedure is shown to be optimally rate adaptive over the entire range of Besov and $L_p$ balls in the sense that it attains certain constrained risk bounds.
Asymptotic efficiency is proved for the constructed in part 1 procedure, i.e. Pinsker's constant is found in the asymptotic lower bound for the minimax quadratic risk. It is shown that the asymptotic minimax quadratic risk of the constructed procedure coincides with this constant.
Estimation of a quadratic functional over parameter spaces that are not quadratically convex is considered. It is shown, in contrast to the theory for quadratically convex parameter spaces, that optimal quadratic rules are often rate suboptimal. In such cases minimax rate optimal procedures are constructed based on local thresholding. These nonquadratic procedures are sometimes fully efficient even when optimal quadratic rules have slow rates of convergence. Moreover, it is shown that when estimating a quadratic functional nonquadratic procedures may exhibit different elbow phenomena than quad…
The paper deals with asymptotic properties of the adaptive procedure proposed in the author paper, 2007, for estimating a unknown nonparametric regression. We prove that this procedure is asymptotically efficient for a quadratic risk, i.e. the asymptotic quadratic risk for this procedure coincides with the Pinsker constant which gives a sharp lower bound for the quadratic risk over all possible estimators.
The paper deals with asymptotic properties of the adaptive procedure proposed in the author paper, 2007, for estimating an unknown nonparametric regression. %\cite{GaPe1}. We prove that this procedure is asymptotically efficient for a quadratic risk, i.e. the asymptotic quadratic risk for this procedure coincides with the Pinsker constant which gives a sharp lower bound for the quadratic risk over all possible estimates
In a circular convolution model, we aim to infer on the density of a circular random variable using observations contaminated by an additive measurement error. We highlight the interplay of the two problems: optimal testing and quadratic functional estimation. Under general regularity assumptions, we determine an upper bound for the minimax risk of estimation for the quadratic functional. The upper bound consists of two terms, one that mimics a classical bias-variance trade-off and a second that causes the typical elbow effect in quadratic functional estimation. Using a minimax optimal estimat…
The paper deals with asymptotic properties of the adaptive procedure proposed in the author paper (2007) for estimation of unknown nonparametric regression. We prove that this procedure is asymptotically efficient for a quadratic risk. It means that the asymptotic quadratic risk for this procedure coincides with a sharp lower bound.
We study distributional properties of a quadratic form of a stationary functional time series under mild moment conditions. As an important application, we obtain consistency rates of estimators of spectral density operators and prove joint weak convergence to a vector of complex Gaussian random operators. Weak convergence is established based on an approximation of the form via transforms of Hilbert-valued martingale difference sequences. As a side-result, the distributional properties of the long-run covariance operator are established.
Independent component analysis (ICA) is linked up with the problem of estimating a non linear functional of a density, for which optimal estimators are well known. The precision of ICA is analyzed from the viewpoint of functional spaces in the wavelet framework. In particular, it is shown that, under Besov smoothness conditions, parametric rate of convergence is achieved by a U-statistic estimator of the wavelet ICA contrast, while the previously introduced plug-in estimator $\hat C^2\_j$, with moderate computational cost, has a rate in $n^{-4s\over 4s+d}$.
We consider the estimation of quadratic functionals in a Gaussian sequence model where the eigenvalues are supposed to be unknown and accessible through noisy observations only. Imposing smoothness assumptions both on the signal and the sequence of eigenvalues, we develop a minimax theory for this problem. We propose a truncated series estimator and show that it attains the optimal rate of convergence if the truncation parameter is chosen appropriately. Consequences for testing problems in inverse problems are equally discussed: in particular, the minimax rates of testing for signal detection …
We consider a quadratic functional regression model in which a scalar response depends on a functional predictor; the common functional linear model is a special case. We wish to test the significance of the nonlinear term in the model. We develop a testing method which is based on projecting the observations onto a suitably chosen finite dimensional space using functional principal component analysis. The asymptotic behavior of our testing procedure is established. A simulation study shows that the testing procedure has good size and power with finite sample sizes. We then apply our test to a…
Motivated by applications in genomics, this paper studies the problem of optimal estimation of a quadratic functional of two normal mean vectors, $Q(μ, θ) = \frac{1}{n}\sum_{i=1}^nμ_i^2θ_i^2$, with a particular focus on the case where both mean vectors are sparse. We propose optimal estimators of $Q(μ, θ)$ for different regimes and establish the minimax rates of convergence over a family of parameter spaces. The optimal rates exhibit interesting phase transitions in this family. The simultaneous signal detection problem is also considered under the minimax framework. It is shown that the propo…
We consider the problem of testing a particular type of composite null hypothesis under a nonparametric multivariate regression model. For a given quadratic functional $Q$, the null hypothesis states that the regression function $f$ satisfies the constraint $Q[f]=0$, while the alternative corresponds to the functions for which $Q[f]$ is bounded away from zero. On the one hand, we provide minimax rates of testing and the exact separation constants, along with a sharp-optimal testing procedure, for diagonal and nonnegative quadratic functionals. We consider smoothness classes of ellipsoidal form…
In this paper, we study estimation of certain integral functionals of one or two densities with samples from stationary m-dependent sequences. We consider two types of U-statistic estimators for these functionals that are functions of the number of epsilon-close vector observations in the samples. We show that the estimators are consistent and obtain their rates of convergence under weak distributional assumptions. In particular, we propose estimators based on incomplete U-statistics which have favorable consistency properties even when m-dependence is the only dependence condition that can be…
We provide general adaptive upper bounds for estimating nonparametric functionals based on second order U-statistics arising from finite dimensional approximation of the infinite dimensional models. We then provide examples of functionals for which the theory produces rate optimally matching adaptive upper and lower bounds. Our results are automatically adaptive in both parametric and nonparametric regimes of estimation and are automatically adaptive and semiparametric efficient in the regime of parametric convergence rate.
We study the adaptive minimax estimation of non-linear integral functionals of a density and extend the results obtained for linear and quadratic functionals to general functionals. The typical rate optimal non-adaptive minimax estimators of "smooth" non-linear functionals are higher order U-statistics. Since Lepski's method requires tight control of tails of such estimators, we bypass such calculations by a modification of Lepski's method which is applicable in such situations. As a necessary ingredient, we also provide a method to control higher order moments of minimax estimator of cubic in…
Statistical inferences for quadratic functionals of linear regression parameter have found wide applications including signal detection, global testing, inferences of error variance and fraction of variance explained. Classical theory based on ordinary least squares estimator works perfectly in the low-dimensional regime, but fails when the parameter dimension $p_n$ grows proportionally to the sample size $n$. In some cases, its performance is not satisfactory even when $n\ge 5p_n$. The main contribution of this paper is to develop {\em dimension-adaptive} inferences for quadratic functionals …
We provide a general theoretical framework to derive Bernstein-von Mises theorems for matrix functionals. The conditions on functionals and priors are explicit and easy to check. Results are obtained for various functionals including entries of covariance matrix, entries of precision matrix, quadratic forms, log-determinant, eigenvalues in the Bayesian Gaussian covariance/precision matrix estimation setting, as well as for Bayesian linear and quadratic discriminant analysis.
For the Gaussian sequence model, we obtain non-asymptotic minimax rates of estimation of the linear, quadratic and the L2-norm functionals on classes of sparse vectors and construct optimal estimators that attain these rates. The main object of interest is the class s-sparse vectors for which we also provide completely adaptive estimators (independent of s and of the noise variance) having only logarithmically slower rates than the minimax ones. Furthermore, we obtain the minimax rates on the Lq-balls where 0 < q < 2. This analysis shows that there are, in general, three zones in the rates of …
We consider a Markov process $X$ associated to a nonnecessarily symmetric Dirichlet form $\mathcal{E}$. We define a stochastic integral with respect to a class of additive functionals of zero quadratic variation and then we obtain an Itô formula for the process $u(X)$, when $u$ is locally in the domain of $\mathcal{E}$.
We consider in this paper a Gaussian sequence model of observations $Y_i$, $i\geq 1$ having mean (or signal) $θ_i$ and variance $σ_i$ which is growing polynomially like $i^γ$, $γ>0$. This model describes a large panel of inverse problems. We estimate the quadratic functional of the unknown signal $\sum_{i\geq 1}θ_i^2$ when the signal belongs to ellipsoids of both finite smoothness functions (polynomial weights $i^α$, $α>0$) and infinite smoothness (exponential weights $e^{βi^r}$, $β>0$, $0<r \leq 2$). We propose a Pinsker type projection estimator in each case and study its quadratic risk. Whe…
We derive an asymptotic expansion for the quadratic variation of a stochastic process satisfying a stochastic differential equation driven by a fractional Brownian motion, based on the theory of asymptotic expansion of Skorohod integrals converging to a mixed normal limit. In order to apply the general theory, it is necessary to estimate functionals that are a randomly weighted sum of products of multiple integrals of the fractional Brownian motion, in expanding the quadratic variation and identifying the limit random symbols. To overcome the difficulty, we introduce two types of exponents by …
This paper considers adaptive, minimax estimation of a quadratic functional in a nonparametric instrumental variables (NPIV) model, which is an important problem in optimal estimation of a nonlinear functional of an ill-posed inverse regression with an unknown operator. We first show that a leave-one-out, sieve NPIV estimator of the quadratic functional can attain a convergence rate that coincides with the lower bound previously derived in Chen and Christensen [2018]. The minimax rate is achieved by the optimal choice of the sieve dimension (a key tuning parameter) that depends on the smoothne…
We consider the problem of constructing Bayesian based confidence sets for linear functionals in the inverse Gaussian white noise model. We work with a scale of Gaussian priors indexed by a regularity hyper-parameter and apply the data-driven (slightly modified) marginal likelihood empirical Bayes method for the choice of this hyper-parameter. We show by theory and simulations that the credible sets constructed by this method have sub-optimal behaviour in general. However, by assuming "self-similarity" the credible sets have rate-adaptive size and optimal coverage. As an application of these r…
We consider the problem of estimation of a linear functional in the Gaussian sequence model where the unknown vector theta in R^d belongs to a class of s-sparse vectors with unknown s. We suggest an adaptive estimator achieving a non-asymptotic rate of convergence that differs from the minimax rate at most by a logarithmic factor. We also show that this optimal adaptive rate cannot be improved when s is unknown. Furthermore, we address the issue of simultaneous adaptation to s and to the variance sigma^2 of the noise. We suggest an estimator that achieves the optimal adaptive rate when both s …
A popular Bayesian nonparametric approach to survival analysis consists in modeling hazard rates as kernel mixtures driven by a completely random measure. In this paper we derive asymptotic results for linear and quadratic functionals of such random hazard rates. In particular, we prove central limit theorems for the cumulative hazard function and for the path-second moment and path-variance of the hazard rate. Our techniques are based on recently established criteria for the weak convergence of single and double stochastic integrals with respect to Poisson random measures. We illustrate our r…
We address the problem of adaptive minimax estimation in white gaussian noise model under $L_p$--loss, $1\leq p\leq\infty,$ on the anisotropic Nikolskii classes. We present the estimation procedure based on a new data-driven selection scheme from the family of kernel estimators with varying bandwidths. For proposed estimator we establish so-called Lp-norm oracle inequality and use it for deriving minimax adaptive results. We prove the existence of rate-adaptive estimators and fully characterize behavior of the minimax risk for different relationships between regularity parameters and norm inde…
Adaptive estimation of linear functionals over a collection of parameter spaces is considered. A between-class modulus of continuity, a geometric quantity, is shown to be instrumental in characterizing the degree of adaptability over two parameter spaces in the same way that the usual modulus of continuity captures the minimax difficulty of estimation over a single parameter space. A general construction of optimally adaptive estimators based on an ordered modulus of continuity is given. The results are complemented by several illustrative examples.
In this paper, we consider a functional linear regression model, where both the covariate and the response variable are functional random variables. We address the problem of optimal nonparametric estimation of the conditional expectation operator in this model. A collection of projection estimators over finite dimensional subspaces is first introduce. We provide a non-asymptotic bias-variance decomposition for the Mean Square Prediction error in the case where these subspaces are generated by the (empirical) PCA functional basis. The automatic trade-off is realized thanks to a model selection…
Covariance function estimation is a fundamental task in multivariate functional data analysis and arises in many applications. In this paper, we consider estimating sparse covariance functions for high-dimensional functional data, where the number of random functions p is comparable to, or even larger than the sample size n. Aided by the Hilbert--Schmidt norm of functions, we introduce a new class of functional thresholding operators that combine functional versions of thresholding and shrinkage, and propose the adaptive functional thresholding estimator by incorporating the variance effects o…
Department of Statistics, Harvard University（哈佛大学统计系）
Recent progress in reinforcement learning has led to remarkable performance in a range of applications, but its deployment in high-stakes settings remains quite rare. One reason is a limited understanding of the behavior of reinforcement algorithms, both in terms of their regret and their ability to learn the underlying system dynamics---existing work is focused almost exclusively on characterizing rates, with little attention paid to the constants multiplying those rates that can be critically important in practice. To start to address this challenge, we study perhaps the simplest non-bandit …
The nonlinear effects of environmental variability on species abundance plays an important role in the maintenance of ecological diversity. Nonetheless, many common models use parametric nonlinear terms pre-determining ecological conclusions. Motivated by this concern, we study the estimate of the second derivative (curvature) of the link function g in a functional single index model. Since the coefficient function and the link function are both unknown, the estimate is expressed as a nested optimization. For a fixed and unknown coefficient function, the link function and its second derivative…
We study estimation of (semi-)inner products between two nonparametric probability distributions, given IID samples from each distribution. These products include relatively well-studied classical $\mathcal{L}^2$ and Sobolev inner products, as well as those induced by translation-invariant reproducing kernels, for which we believe our results are the first. We first propose estimators for these quantities, and the induced (semi)norms and (pseudo)metrics. We then prove non-asymptotic upper bounds on their mean squared error, in terms of weights both of the inner product and of the two distribut…
