We propose an estimator for the mean of random variables in separable real Banach spaces using the empirical characteristic function. Assuming that the covariance operator of the random variable is bounded in a precise sense, we show that the proposed estimator achieves the optimal sub-Gaussian rate, except for a faster decaying mean-dependent term. Under a mild condition, an iterative refinement procedure can essentially eliminate the mean-dependent term and provide a shift-equivariant estimate. Our results particularly suggests that a certain Gaussian width that appears in the best known rate in the literature might not be necessary. Furthermore, using the boundedness of the characteristic functions, we also show that, except possibly at high signal-to-noise ratios, the proposed estimator is gracefully robust against adversarial "contamination". Our analysis is overall concise and transparent, thanks to the tractability of the characteristic functions.
This work is concerned with the study of asymptotic properties of nonparametric density estimates in the framework of circular data. The estimation procedure here applied is based on wavelet thresholding methods: the wavelets used are the so-called Mexican needlets, which describe a nearly-tight frame on the circle. We study the asymptotic behaviour of the $L^{2}$-risk function for these estimates, in particular its adaptivity, proving that its rate of convergence is nearly optimal.
In this paper, we propose a method to build a sequence of generalized empirical principal curves, with selected length, so that, in Hausdor distance, the images of the estimating principal curves converge in probability to the image of g.
In this article, we introduce a Gegenbauer autoregressive tempered fractionally integrated moving average (GARTFIMA) process. We work on the spectral density and autocovariance function for the introduced process. The parameter estimation is done using the empirical spectral density with the help of the nonlinear least square technique and the Whittle likelihood estimation technique. The performance of the proposed estimation techniques is assessed on simulated data. Further, the introduced process is shown to better model the real-world data in comparison to other time series models.
We consider the problem of empirical Bayes estimation of multiple variances when provided with sample variances. Assuming an arbitrary prior on the variances, we derive different versions of the Bayes estimators using different loss functions. For one particular loss function, the resulting Bayes estimator relies on the marginal cumulative distribution function of the sample variances only. When replacing it with the empirical distribution function, we obtain an empirical Bayes version called F-modeling based empirical Bayes estimator of variances. We provide theoretical properties of this est…
Nearly optimal minimax estimator for high-dimensional sparse linear regression
我们提出了一个很好的研究统计估计问题的估计：估计的线性回归模型与软稀疏约束（$\ell_q$约束与$0<q\leq1$）在高维设置。我们首先提出了一个家庭的估计，称为投影最近邻估计，并显示，通过使用凸几何的结果，这种估计是在一个对数因子的最佳任何设计矩阵。然后利用[SIAM J. COMPUT. 36（2007）1764-1776]，我们获得了用于计算任何这样的估计任务的极大极小风险的近似算法，并且还获得了用于$\ell_1$稀疏约束的重要情况的多项式时间近似最优估计器。尽管对这个问题进行了数十年的研究，但以前只知道特殊情况下的结果。当参数半径未知时，我们也将该方法推广到自适应的情况。
In this article, by using composite asymmetric least squares (CALS) and empirical likelihood, we propose a two-step procedure to estimate the conditional value at risk (VaR) and conditional expected shortfall (ES) for the GARCH series. First, we perform asymmetric least square regressions at several significance levels to model the volatility structure and separate it from the innovation process in the GARCH model. Note that expectile can serve as a bond to make up the gap from VaR estimation to ES estimation because there exists a bijective mapping from expectiles to specific quantile, and ES…
The aim of this paper is to present a new estimation procedure that can be applied in many statistical frameworks including density and regression and which leads to both robust and optimal (or nearly optimal) estimators. In density estimation, they asymptotically coincide with the celebrated maximum likelihood estimators at least when the statistical model is regular enough and contains the true density to estimate. For very general models of densities, including non-compact ones, these estimators are robust with respect to the Hellinger distance and converge at optimal rate (up to a possible…
In this work, we delve into the nonparametric empirical Bayes theory and approximate the classical Bayes estimator by a truncation of the generalized Laguerre series and then estimate its coefficients by minimizing the prior risk of the estimator. The minimization process yields a system of linear equations the size of which is equal to the truncation level. We focus on the empirical Bayes estimation problem when the mixing distribution, and therefore the prior distribution, has a support on the positive real half-line or a subinterval of it. By investigating several common mixing distribution…
Classification and estimation in the Stochastic Block Model based on the empirical degrees
随机块模型（Holland et al.，1983）是一种异构网络数据的混合模型。与通常的统计框架不同，新节点提供了关于该模型中先前节点的额外信息。因此，度的分布集中在有条件的节点类的点。我们表明，在一个温和的假设下，分类，估计和模型选择实际上可以实现不超过经验度数据。我们提供了一个算法，能够处理非常大的网络和一致的估计基于它。特别是，我们证明了至少一个节点的误分类的概率的界，包括当类的数量增长。
The Pitman-Yor process is a random discrete probability distribution of which the atoms can be used to model the relative abundance of species. The process is indexed by a type parameter $σ$, which controls the number of different species in a finite sample from a realization of the distribution. A random sample of size $n$ from the Pitman-Yor process of type $σ>0$ will contain of the order $n^σ$ distinct values (``species''). In this paper we consider the estimation of the type parameter by both empirical Bayes and full Bayes methods. We derive the asymptotic normality of the empirical Bayes …
In this article, we construct empirical likelihood (EL)-weighted estimators of linear functionals of a probability measure in the presence of side information. Motivated by nuisance parameters in semiparametric models with possibly infinite dimensions, we consider the use of estimated constraint functions and allow the number of constraints to grow with the sample size. We study the asymptotic properties and efficiency gains. The results are used to construct improved estimators of parameters in structural equation models. The EL-weighted estimators of parameters are shown to have reduced vari…
We construct an algorithm, running in time $\tilde{\mathcal O}(N d + uK d)$, which is robust to outliers and heavy-tailed data and which achieves the subgaussian rate from [Lugosi, Mendelson] \begin{equation}\label{eq:intro_subgaus_rate} \sqrt{\frac{{\rm Tr}(Σ)}{N}}+\sqrt{\frac{||Σ||_{op}K}{N}} \end{equation}with probability at least $1-\exp(-c_0K)-\exp(-c_1 u)$ where $Σ$ is the covariance matrix of the informative data, $K\in\{1, \ldots, K\}$ is some parameter (number of block means) and $u>0$ is another parameter of the algorithm. This rate is achieved when $K\geq c_1 |\mathcal O|$ where $|\…
Duke University（杜克大学）；University of Southern California（南加州大学）
We study the fundamental problem of high-dimensional mean estimation in a robust model where a constant fraction of the samples are adversarially corrupted. Recent work gave the first polynomial time algorithms for this problem with dimension-independent error guarantees for several families of structured distributions. In this work, we give the first nearly-linear time algorithms for high-dimensional robust mean estimation. Specifically, we focus on distributions with (i) known covariance and sub-gaussian tails, and (ii) unknown bounded covariance. Given $N$ samples on $\mathbb{R}^d$, an $ε$-…
This paper studies the asymptotic properties of and alternative inference methods for kernel density estimation (KDE) for dyadic data. We first establish uniform convergence rates for dyadic KDE. Secondly, we propose a modified jackknife empirical likelihood procedure for inference. The proposed test statistic is asymptotically pivotal regardless of presence of dyadic clustering. The results are further extended to cover the practically relevant case of incomplete dyadic data. Simulations show that this modified jackknife empirical likelihood-based inference procedure delivers precise coverage…
We propose an empirical estimator of the preferential attachment function $f$ in the setting of general preferential attachment trees. Using a supercritical continuous-time branching process framework, we prove the almost sure consistency of the proposed estimator. We perform simulations to study the empirical properties of our estimators.
We consider a problem of manifold estimation from noisy observations. Many manifold learning procedures locally approximate a manifold by a weighted average over a small neighborhood. However, in the presence of large noise, the assigned weights become so corrupted that the averaged estimate shows very poor performance. We suggest a structure-adaptive procedure, which simultaneously reconstructs a smooth manifold and estimates projections of the point cloud onto this manifold. The proposed approach iteratively refines the weights on each step, using the structural information obtained at previ…
We design a new, fast algorithm for agnostically learning univariate probability distributions whose densities are well approximated by piecewise polynomial functions. Let $f$ be the density function of an arbitrary univariate distribution, and suppose that $f$ is $\mathrm{OPT}$-close in $L_1$-distance to an unknown piecewise polynomial function with $t$ interval pieces and degree $d$. Our algorithm draws $n = O(t(d+1)/ε^2)$ samples from $f$, runs in time $\tilde{O}(n \cdot \mathrm{poly}(d))$, and with probability at least $9/10$ outputs an $O(t)$-piecewise degree-$d$ hypothesis $h$ that is $4…
Microsoft Research AI（微软研究院人工智能部门）；University of Washington（华盛顿大学）
Robust covariance estimation is the following, well-studied problem in high dimensional statistics: given $N$ samples from a $d$-dimensional Gaussian $\mathcal{N}(\boldsymbol{0}, Σ)$, but where an $\varepsilon$-fraction of the samples have been arbitrarily corrupted, output $\widehatΣ$ minimizing the total variation distance between $\mathcal{N}(\boldsymbol{0}, Σ)$ and $\mathcal{N}(\boldsymbol{0}, \widehatΣ)$. This corresponds to learning $Σ$ in a natural affine-invariant variant of the Frobenius norm known as the \emph{Mahalanobis norm}. Previous work of Cheng et al demonstrated an algorithm …
Due to their conjugate posteriors, Gaussian process priors are attractive for estimating the drift of stochastic differential equations with continuous time observations. However, their performance strongly depends on the choice of the hyper-parameters. We employ the marginal maximum likelihood estimator to estimate the scaling and/or smoothness parameter(s) of the prior and show that the corresponding posterior has optimal rates of convergence. General theorems do not apply directly to this model as the usual test functions are with respect to a random Hellinger-type metric. We allow for cont…
University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Shanghai University of Finance and Economics（上海财经大学）；Institute of Advanced Study（高等研究院）；Shanghai Jiao Tong University（上海交通大学）
We study the problem of learning Bayesian networks where an $ε$-fraction of the samples are adversarially corrupted. We focus on the fully-observable case where the underlying graph structure is known. In this work, we present the first nearly-linear time algorithm for this problem with a dimension-independent error guarantee. Previous robust algorithms with comparable error guarantees are slower by at least a factor of $(d/ε)$, where $d$ is the number of variables in the Bayesian network and $ε$ is the fraction of corrupted samples. Our algorithm and analysis are considerably simpler than tho…
We study the asymptotic normality of two feasible estimators of the integrated volatility of volatility based on the Fourier methodology, which does not require the pre-estimation of the spot volatility. We show that the bias-corrected estimator reaches the optimal rate $n^{1/4}$, while the estimator without bias-correction has a slower convergence rate and a smaller asymptotic variance. Additionally, we provide simulation results that support the theoretical asymptotic distribution of the rate-efficient estimator and show the accuracy of the latter in comparison with a rate-optimal estimator …
Functional Generalized Empirical Likelihood Estimation for Conditional Moment Restrictions
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Weierstrass Institute for Applied Analysis and Stochastics（魏尔斯特拉斯应用分析与随机过程研究所）；Eidgenössische Technische Hochschule Zürich（苏黎世联邦理工学院）
This paper presents a detailed theoretical analysis of the three stochastic approximation proximal gradient algorithms proposed in our companion paper [49] to set regularization parameters by marginal maximum likelihood estimation. We prove the convergence of a more general stochastic approximation scheme that includes the three algorithms of [49] as special cases. This includes asymptotic and non-asymptotic convergence results with natural and easily verifiable conditions, as well as explicit bounds on the convergence rates. Importantly, the theory is also general in that it can be applied to…
