Estimation of a quadratic functional over parameter spaces that are not quadratically convex is considered. It is shown, in contrast to the theory for quadratically convex parameter spaces, that optimal quadratic rules are often rate suboptimal. In such cases minimax rate optimal procedures are constructed based on local thresholding. These nonquadratic procedures are sometimes fully efficient even when optimal quadratic rules have slow rates of convergence. Moreover, it is shown that when estimating a quadratic functional nonquadratic procedures may exhibit different elbow phenomena than quadratic procedures.
Adaptive estimation of a quadratic functional over both Besov and $L_p$ balls is considered. A collection of nonquadratic estimators are developed which have useful bias and variance properties over individual Besov and $L_p$ balls. An adaptive procedure is then constructed based on penalized maximization over this collection of nonquadratic estimators. This procedure is shown to be optimally rate adaptive over the entire range of Besov and $L_p$ balls in the sense that it attains certain constrained risk bounds.
The article is devoted to the nonparametric estimation of the quadratic covariation of non-synchronously observed Itô processes in an additive microstructure noise model. In a high-frequency setting, we aim at establishing an asymptotic distribution theory for a generalized multiscale estimator including a feasible central limit theorem with optimal convergence rate on convenient regularity assumptions. The inevitably remaining impact of asynchronous deterministic sampling schemes and noise corruption on the asymptotic distribution is precisely elucidated. A case study for various important ex…
In a previous article, a least square regression estimation procedure was proposed: first, we condiser a family of functions and study the properties of an estimator in every unidimensionnal model defined by one of these functions; we then show how to aggregate these estimators. The purpose of this paper is to extend this method to the case of density estimation. We first give a general overview of the method, adapted to the density estimation problem. We then show that this leads to adaptative estimators, that means that the estimator reaches the best possible rate of convergence (up to a $\l…
We prove conditional asymptotic normality of a class of quadratic U-statistics that are dominated by their degenerate second order part and have kernels that change with the number of observations. These statistics arise in the construction of estimators in high-dimensional semi- and non-parametric models, and in the construction of nonparametric confidence sets. This is illustrated by estimation of the integral of a square of a density or regression function, and estimation of the mean response with missing data. We show that estimators are asymptotically normal even in the case that the rate…
The vanilla method in univariate extreme-value theory consists of fitting the three-parameter Generalized Extreme-Value (GEV) distribution to a sample of block maxima. Despite claims to the contrary, the asymptotic normality of the maximum likelihood estimator has never been established. In this paper, a formal proof is given using a general result on the maximum likelihood estimator for parametric families that are differentiable in quadratic mean but whose supports depend on the parameter. An interesting side result concerns the (lack of) differentiability in quadratic mean of the GEV family…
The author uses a Stein-type covariance identity to obtain moment estimators for the parameters of the quadratic polynomial subfamily of Pearson distributions. The asymptotic distribution of the estimators is obtained, and normality and symmetry tests based on it are provided. Simulation is used to compare the performance of the proposed tests with that of other existing tests for symmetry and normality.
We study parametric inference for diffusion processes when observations occur nonsynchronously and are contaminated by market microstructure noise. We construct a quasi-likelihood function and study asymptotic mixed normality of maximum-likelihood- and Bayes-type estimators based on it. We also prove the local asymptotic normality of the model and asymptotic efficiency of our estimator when the diffusion coefficients are constant and noise follows a normal distribution. We conjecture that our estimator is asymptotically efficient even when the latent process is a general diffusion process. An …
We consider linear processes, not necessarily Gaussian, with long, short or negative memory. The memory parameter is estimated semi-parametrically using wavelets from a sample $X_1,...,X_n$ of the process. We treat both the log-regression wavelet estimator and the wavelet Whittle estimator. We show that these estimators are asymptotically normal as the sample size $n\to\infty$ and we obtain an explicit expression for the limit variance. These results are derived from a general result on the asymptotic normality of the empirical scalogram for linear processes, conveniently centered and normaliz…
This article introduces a method for estimating the smoothness of a stationary, isotropic Gaussian random field from irregularly spaced data. This involves novel constructions of higher-order quadratic variations and the establishment of the corresponding fixed-domain asymptotic theory. In particular, we consider: (i) higher-order quadratic variations using nonequispaced line transect data, (ii) second-order quadratic variations from a sample of Gaussian random field observations taken along a smooth curve in ${\mathbb{R}}^2$, (iii) second-order quadratic variations based on deformed lattice d…
Motivated by applications in genomics, this paper studies the problem of optimal estimation of a quadratic functional of two normal mean vectors, $Q(μ, θ) = \frac{1}{n}\sum_{i=1}^nμ_i^2θ_i^2$, with a particular focus on the case where both mean vectors are sparse. We propose optimal estimators of $Q(μ, θ)$ for different regimes and establish the minimax rates of convergence over a family of parameter spaces. The optimal rates exhibit interesting phase transitions in this family. The simultaneous signal detection problem is also considered under the minimax framework. It is shown that the propo…
The Rényi entropy is a generalization of the Shannon entropy and is widely used in mathematical statistics and applied sciences for quantifying the uncertainty in a probability distribution. We consider estimation of the quadratic Rényi entropy and related functionals for the marginal distribution of a stationary m-dependent sequence. The U-statistic estimators under study are based on the number of epsilon-close vector observations in the corresponding sample. A variety of asymptotic properties for these estimators are obtained (e.g., consistency, asymptotic normality, Poisson convergence). T…
In this paper we prove large and moderate deviations principles for the recursive kernel estimator of a probability density function and its partial derivatives. Unlike the density estimator, the derivatives estimators exhibit a quadratic behavior not only for the moderate deviations scale but also for the large deviations one. We provide results both for the pointwise and the uniform deviations.
This paper presents a central limit theorem for a pre-averaged version of the realized covariance estimator for the quadratic covariation of a discretely observed semimartingale with noise. The semimartingale possibly has jumps, while the observation times show irregularity, non-synchronicity, and some dependence on the observed process. It is shown that the observation times' effect on the asymptotic distribution of the estimator is only through two characteristics: the observation frequency and the covariance structure of the noise. This is completely different from the case of the realized …
The aim of this work is to estimate a quadratic functional of a unknown Wigner function from noisy tomographic data. The Wigner function can be seen as the representation of the quantum state of a light beam. The estimation of a quadratic functional is done from result of quantum homodyne measurement performed on identically prepared quantum systems. We start by constructing an estimator of a quadratic functional of the Wigner function. We show that the proposed estimator is optimal or nearly optimal in a minimax sense over a class of infinitely differentiable functions. Parametric rates are a…
We derive convenient uniform concentration bounds and finite sample multivariate normal approximation results for quadratic forms, then describe some applications involving variance components estimation in linear random-effects models. Random-effects models and variance components estimation are classical topics in statistics, with a corresponding well-established asymptotic theory. However, our finite sample results for quadratic forms provide additional flexibility for easily analyzing random-effects models in non-standard settings, which are becoming more important in modern applications (…
The paper concerns the asymptotic distribution of the mixture density estimator, proposed by Oppenheim et al 2006, in the aggregation/disaggregation problem of random parameter AR(1) process. We prove that, under mild conditions on the (semiparametric) form of the mixture density, the estimator is asymptotically normal. The proof is based on the limit theory for the quadratic form in linear random variables developed by Bhansali et al 2007. The moving average representation of the aggregated process is investigated. A small simulation study illustrates the result.
We consider the convolution model where i.i.d. random variables $X_i$ having unknown density $f$ are observed with additive i.i.d. noise, independent of the $X$'s. We assume that the density $f$ belongs to either a Sobolev class or a class of supersmooth functions. The noise distribution is known and its characteristic function decays either polynomially or exponentially asymptotically. We consider the problem of goodness-of-fit testing in the convolution model. We prove upper bounds for the risk of a test statistic derived from a kernel estimator of the quadratic functional $\int f^2$ based o…
In this paper, we study the Edgeworth expansion for a pre-averaging estimator of quadratic variation in the framework of continuous diffusion models observed with noise. More specifically, we obtain a second order expansion for the joint density of the estimators of quadratic variation and its asymptotic variance. Our approach is based on martingale embedding, Malliavin calculus and stable central limit theorems for continuous diffusions. Moreover, we derive the density expansion for the studentized statistic, which might be applied to construct asymptotic confidence regions.
We consider estimation of the quadratic (co)variation of a semimartingale from discrete observations which are irregularly spaced under high-frequency asymptotics. In the univariate setting, results by Jacod (2008) are generalized to the case of irregular observations. In the two-dimensional setup under non-synchronous observations, we derive a stable central limit theorem for the Hayashi-Yoshida estimator in the presence of jumps. We reveal how idiosyncratic and simultaneous jumps affect the asymptotic distribution. Observation times generated by Poisson processes are explicitly discussed.
This paper is concerned with adaptive nonparametric estimation using the Goldenshluger-Lepski selection method. This estimator selection method is based on pairwise comparisons between estimators with respect to some loss function. The method also involves a penalty term that typically needs to be large enough in order that the method works (in the sense that one can prove some oracle type inequality for the selected estimator). In the case of density estimation with kernel estimators and a quadratic loss, we show that the procedure fails if the penalty term is chosen smaller than some critica…
A new nonparametric estimator of the local Hurst function of a multifractional Gaussian process based on the increment ratio (IR) statistic is defined. In a general frame, the point-wise and uniform weak and strong consistency and a multidimensional central limit theorem for this estimator are established. Similar results are obtained for a refinement of the generalized quadratic variations (QV) estimator. The example of the multifractional Brownian motion is studied in detail. A simulation study is included showing that the IR-estimator is more accurate than the QV-estimator.
This paper proposes a local representation for Empirical Likelihood (EL). EL admits the classical local linear quadratic representation by its likelihood ratio property. A local estimator is derived by using the new representation. Consistency, local asymptotic normality, and asymptotic optimality results hold for the new estimator. In particular, when the regularity conditions do not include any differentiability assumption, these asymptotic results are still valid for the local estimator. Simulations illustrate that the local method improves the inference accuracy of EL.
The partially observed linear Gaussian system of stochastic differential equations with low noise in observations is considered. A kernel-type estimators are used for estimation of the quadratic variation of the derivative of the limit of the observed process. Then this estimator is used for nonparametric estimation of the integral of the square of volatility of unobservable component. This estimator is also used for construction of substitution estimators in the case where the drift in observable component and the volatility of the state component depend on some unknown parameter. Then this s…
Consider estimating the n by p matrix of means of an n by p matrix of independent normally distributed observations with constant variance, where the performance of an estimator is judged using a p by p matrix quadratic error loss function. A matrix version of the James-Stein estimator is proposed, depending on a tuning constant. It is shown to dominate the usual maximum likelihood estimator for some choices of of the tuning constant when n is greater than or equal to 3. This result also extends to other shrinkage estimators and settings.
We consider the estimation of quadratic functionals in a Gaussian sequence model where the eigenvalues are supposed to be unknown and accessible through noisy observations only. Imposing smoothness assumptions both on the signal and the sequence of eigenvalues, we develop a minimax theory for this problem. We propose a truncated series estimator and show that it attains the optimal rate of convergence if the truncation parameter is chosen appropriately. Consequences for testing problems in inverse problems are equally discussed: in particular, the minimax rates of testing for signal detection …
In this paper we introduce a general method for estimating the quadratic covariation of one or more spot parameters processes associated with continuous time semimartingales. This estimator is applicable to a wide range of spot parameter processes, and may also be used to estimate the leverage effect of stochastic volatility models. The estimator we introduce is based on sums of squared increments of second differences of the observed process, and the intervals over which the differences are computed are rolling and overlapping. This latter feature lets us take full advantage of the data, and,…
In this paper we give a central limit theorem for the weighted quadratic variations process of a two-parameter Brownian motion. As an application, we show that the discretized quadratic variations $\sum_{i=1}^{[n s]} \sum_{j=1}^{[n t]} | Δ_{i,j} Y |^2$ of a two-parameter diffusion $Y=(Y_{(s,t)})_{(s,t)\in[0,1]^2}$ observed on a regular grid $G_n$ is an asymptotically normal estimator of the quadratic variation of $Y$ as $n$ goes to infinity.
This paper studies the behaviour of quadratic variations of a stochastic wave equation driven by a noise that is white in space and fractional in time. Complementing the analysis of quadratic variations in the space component carried out by M. Khalil and C. A. Tudor (2018) and by R. Shevchenko, M. Slaoui and C. A. Tudor (2020), it focuses on the time component of the solution process. For different values of the Hurst parameter a central and a noncentral limit theorems are proved, allowing to construct consistent parameter estimators and compare them to the finding in the space-dependent case.…
We study spectral density estimation under local differential privacy. Anonymization is achieved through truncation followed by Laplace perturbation. We select our estimator from a set of candidate estimators by a penalized contrast criterion. This estimator is shown to attain nearly the same rate of convergence as the best estimator from the candidate set. A key ingredient of the proof are recent results on concentration of quadratic forms in terms of sub-exponential random variables obtained in arXiv:1903.05964. We illustrate our findings in a small simulation study.
This paper considers adaptive, minimax estimation of a quadratic functional in a nonparametric instrumental variables (NPIV) model, which is an important problem in optimal estimation of a nonlinear functional of an ill-posed inverse regression with an unknown operator. We first show that a leave-one-out, sieve NPIV estimator of the quadratic functional can attain a convergence rate that coincides with the lower bound previously derived in Chen and Christensen [2018]. The minimax rate is achieved by the optimal choice of the sieve dimension (a key tuning parameter) that depends on the smoothne…
We consider the problem of efficient estimation for the drift of fractional Brownian motion $B^H:=(B^H_t)_{t\in[0,T]}$ with hurst parameter $H$ less than 1/2. We also construct superefficient James-Stein type estimators which dominate, under the usual quadratic risk, the natural maximum likelihood estimator.
The Garman-Klass unbiased estimator of the variance per unit time of a zero-drift Brownian Motion B, based on the usual financial data that reports for time windows of equal length the open (OPEN), minimum (MIN), maximum (MAX) and close (CLOSE) values, is quadratic in the statistic S1=(CLOSE-OPEN, OPEN-MIN, MAX-OPEN). This estimator, with efficiency 7.4 with respect to the classical estimator (CLOSE-OPEN)^2, is widely believed to be of minimal variance. The current report disproves this belief by exhibiting an unbiased estimator with slightly but strictly higher efficiency 7.7322. The essence …
We show that the generalized method of moments (GMM) estimation problem in instrumental variable quantile regression (IVQR) models can be equivalently formulated as a mixed integer quadratic programming problem. This enables exact computation of the GMM estimators for the IVQR models. We illustrate the usefulness of our algorithm via Monte Carlo experiments and an application to demand for fish.
The problem of estimating the spectral density matrix $f(w)$ of a multivariate time series is revisited with special focus on the frequencies $w=0$ and $w=π$. Recognizing that the entries of the spectral density matrix at these two boundary points are real-valued, we propose a new estimator constructed from a local polynomial regression of the real portion of the multivariate periodogram. The case $w=0$ is of particular importance, since $f(0)$ is associated with the large-sample covariance matrix of the sample mean; hence, estimating $f(0)$ is crucial in order to conduct any sort of statistic…
The Kaplan--Meier (KM) estimator, which provides a nonparametric estimate of a survival function for time-to-event data, has wide application in clinical studies, engineering, economics and other fields. The theoretical properties of the KM estimator including its consistency and asymptotic distribution have been extensively studied. We reconstruct the KM estimator as an M-estimator by maximizing a quadratic M-function based on concordance, which can be computed using the expectation--maximization (EM) algorithm. It is shown that the convergent point of the EM algorithm coincides with the trad…
We consider the problem of estimating the proportion $θ$ of true null hypotheses in a multiple testing context. The setup is classically modeled through a semiparametric mixture with two components: a uniform distribution on interval $[0,1]$ with prior probability $θ$ and a nonparametric density $f$. We discuss asymptotic efficiency results and establish that two different cases occur whether $f$ vanishes on a set with non null Lebesgue measure or not. In the first case, we exhibit estimators converging at parametric rate, compute the optimal asymptotic variance and conjecture that no estimato…
We propose a new estimator, the quadratic form estimator, of the Kronecker product model for covariance matrices. We show that this estimator has good properties in the large dimensional case (i.e., the cross-sectional dimension $n$ is large relative to the sample size $T$). In particular, the quadratic form estimator is consistent in a relative Frobenius norm sense provided $\log^3n/T\to 0$. We obtain the limiting distributions of Lagrange multiplier (LM) and Wald tests under both the null and local alternatives concerning the mean vector $μ$. Testing linear restrictions of $μ$ is also invest…
Independent component analysis (ICA) is linked up with the problem of estimating a non linear functional of a density, for which optimal estimators are well known. The precision of ICA is analyzed from the viewpoint of functional spaces in the wavelet framework. In particular, it is shown that, under Besov smoothness conditions, parametric rate of convergence is achieved by a U-statistic estimator of the wavelet ICA contrast, while the previously introduced plug-in estimator $\hat C^2\_j$, with moderate computational cost, has a rate in $n^{-4s\over 4s+d}$.
Partial coherence is an important quantity derived from spectral or precision matrices and is used in seismology, meteorology, oceanography, neuroscience and elsewhere. If the number of complex degrees of freedom only slightly exceeds the dimension of the multivariate stationary time series, spectral matrices are poorly conditioned and shrinkage techniques suggest themselves. When true partial coherencies are quite large then for shrinkage estimators of the diagonal weighting kind it is shown empirically that the minimization of risk using quadratic loss (QL) leads to oracle partial coherence …
In this paper, we study estimation of certain integral functionals of one or two densities with samples from stationary m-dependent sequences. We consider two types of U-statistic estimators for these functionals that are functions of the number of epsilon-close vector observations in the samples. We show that the estimators are consistent and obtain their rates of convergence under weak distributional assumptions. In particular, we propose estimators based on incomplete U-statistics which have favorable consistency properties even when m-dependence is the only dependence condition that can be…
This paper discusses the simultaneous inference of mean parameters in a family of distributions with quadratic variance function. We first introduce a class of semiparametric/parametric shrinkage estimators and establish their asymptotic optimality properties. Two specific cases, the location-scale family and the natural exponential family with quadratic variance function, are then studied in detail. We conduct a comprehensive simulation study to compare the performance of the proposed methods with existing shrinkage estimators. We also apply the method to real data and obtain encouraging resu…
We propose a new penalized method for variable selection and estimation that explicitly incorporates the correlation patterns among predictors. This method is based on a combination of the minimax concave penalty and Laplacian quadratic associated with a graph as the penalty function. We call it the sparse Laplacian shrinkage (SLS) method. The SLS uses the minimax concave penalty for encouraging sparsity and Laplacian quadratic penalty for promoting smoothness among coefficients associated with the correlated predictors. The SLS has a generalized grouping property with respect to the graph rep…
The paper deals with asymptotic properties of the adaptive procedure proposed in the author paper, 2007, for estimating a unknown nonparametric regression. We prove that this procedure is asymptotically efficient for a quadratic risk, i.e. the asymptotic quadratic risk for this procedure coincides with the Pinsker constant which gives a sharp lower bound for the quadratic risk over all possible estimators.
We study distributional properties of a quadratic form of a stationary functional time series under mild moment conditions. As an important application, we obtain consistency rates of estimators of spectral density operators and prove joint weak convergence to a vector of complex Gaussian random operators. Weak convergence is established based on an approximation of the form via transforms of Hilbert-valued martingale difference sequences. As a side-result, the distributional properties of the long-run covariance operator are established.
We study estimation of (semi-)inner products between two nonparametric probability distributions, given IID samples from each distribution. These products include relatively well-studied classical $\mathcal{L}^2$ and Sobolev inner products, as well as those induced by translation-invariant reproducing kernels, for which we believe our results are the first. We first propose estimators for these quantities, and the induced (semi)norms and (pseudo)metrics. We then prove non-asymptotic upper bounds on their mean squared error, in terms of weights both of the inner product and of the two distribut…
Co-heritability is an important concept that characterizes the genetic associations within pairs of quantitative traits. There has been significant recent interest in estimating the co-heritability based on data from the genome-wide association studies (GWAS). This paper introduces two measures of co-heritability in the high-dimensional linear model framework, including the inner product of the two regression vectors and a normalized inner product by their lengths. Functional de-biased estimators (FDEs) are developed to estimate these two co-heritability measures. In addition, estimators of qu…
In this paper optimal experimental designs for inverse quadratic regression models are determined. We consider two different parameterizations of the model and investigate local optimal designs with respect to the $c$-, $D$- and $E$-criteria, which reflect various aspects of the precision of the maximum likelihood estimator for the parameters in inverse quadratic regression models. In particular it is demonstrated that for a sufficiently large design space geometric allocation rules are optimal with respect to many optimality criteria. Moreover, in numerous cases the designs with respect to th…
