Alisa Kirichenko, Harry van Zanten
We study minimax lower bounds for function estimation problems on large graph when the target function is smoothly varying over the graph. We derive minimax rates in the context of regression and classification problems on graphs that satisfy an asymptotic shape assumption and with a smoothness condition on the target function, both formulated in terms of the graph Laplacian.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
In this paper, we derive minimax rates for estimating both parametric and nonparametric components in partially linear additive models with high dimensional sparse vectors and smooth functional components. The minimax lower bound for Euclidean components is the typical sparse estimation rate that is independent of nonparametric smoothness indices. However, the minimax lower bound for each component function exhibits an interplay between the dimensionality and sparsity of the parametric component and the smoothness of the relevant nonparametric component. Indeed, the minimax risk for smooth non…
We prove that under some regularity and strong identifiability conditions, around a mixing distribution with $m_0$ components, the optimal local minimax rate of estimation of a mixture with $m$ components is $n^{-1/(4(m-m_0) + 2)}$. This corrects a previous paper by Chen (1995) in The Annals of Statistics.
We address the problem of adaptive minimax density estimation on $\bR^d$ with $\bL_p$--loss on the anisotropic Nikol'skii classes. We fully characterize behavior of the minimax risk for different relationships between regularity parameters and norm indexes in definitions of the functional class and of the risk. In particular, we show that there are four different regimes with respect to the behavior of the minimax risk. We develop a single estimator which is (nearly) optimal in orderover the complete scale of the anisotropic Nikol'skii classes. Our estimation procedure is based on a data-drive…
In this paper, we observe a sparse mean vector through Gaussian noise and we aim at estimating some additive functional of the mean in the minimax sense. More precisely, we generalize the results of (Collier et al., 2017, 2019) to a very large class of functionals. The optimal minimax rate is shown to depend on the polynomial approximation rate of the marginal functional, and optimal estimators achieving this rate are built.
We derive non-asymptotic minimax bounds for the Hausdorff estimation of $d$-dimensional submanifolds $M \subset \mathbb{R}^D$ with (possibly) non-empty boundary $\partial M$. The model reunites and extends the most prevalent $\mathcal{C}^2$-type set estimation models: manifolds without boundary, and full-dimensional domains. We consider both the estimation of the manifold $M$ itself and that of its boundary $\partial M$ if non-empty. Given $n$ samples, the minimax rates are of order $O\bigl((\log n/n)^{2/d}\bigr)$ if $\partial M = \emptyset$ and $O\bigl((\log n/n)^{2/(d+1)}\bigr)$ if $\partial…
This paper deals with minimax rates of convergence for estimation of density functions on the real line. The densities are assumed to be location mixtures of normals, a global regularity requirement that creates subtle difficulties for the application of standard minimax lower bound methods. Using novel Fourier and Hermite polynomial techniques, we determine the minimax optimal rate - slightly larger than the parametric rate - under squared error loss. For Hellinger loss, we provide a minimax lower bound using ideas modified from the squared error loss case.
For the problem of nonparametric estimation of signal in Gaussian noise we point out the strong asymptotically minimax estimators on maxisets for linear estimators (see \cite{ker93,rio}). It turns out that the order of rates of convergence of Pinsker estimator on this maxisets is worse than the order of rates of convergence for the class of linear estimators considered on this maxisets. We show that balls in Sobolev spaces are maxisets for Pinsker estimators.
Minimax density estimation in the adversarial framework under local differential privacy
本文研究对抗框架下局部差分隐私约束的非参数密度估计问题，提出Coordinate block privacy机制与投影估计器结合实现极小化极大最优速率，并开发了对数项内最优的自适应程序。
We establish minimax optimal rates of convergence for nonparametric estimation in functional ANOVA models when data from first-order partial derivatives are available. Our results reveal that partial derivatives can improve convergence rates for function estimation with deterministic or random designs. In particular, for full $d$-interaction models, the optimal rates with first-order partial derivatives on $p$ covariates are identical to those for $(d-p)$-interaction models without partial derivatives. For additive models, the rates by using all first-order partial derivatives are root-$n$ to …
We study nonparametric density estimation problems where error is measured in the Wasserstein distance, a metric on probability distributions popular in many areas of statistics and machine learning. We give the first minimax-optimal rates for this problem for general Wasserstein distances, and show that, unlike classical nonparametric density estimation, these rates depend on whether the densities in question are bounded below. Motivated by variational problems involving the Wasserstein distance, we also show how to construct discretely supported measures, suitable for computational purposes,…
Estimation of convex functions finds broad applications in engineering and science, while convex shape constraint gives rise to numerous challenges in asymptotic performance analysis. This paper is devoted to minimax optimal estimation of univariate convex functions from the Hölder class in the framework of shape constrained nonparametric estimation. Particularly, the paper establishes the optimal rate of convergence in two steps for the minimax sup-norm risk of convex functions with the Hölder order between one and two. In the first step, by applying information theoretical results on probabi…
Minimax Private Estimation of Smooth Optimal-Transport Maps
针对差分隐私约束下光滑最优传输映射的估计问题，提出基于小波的密度估计器和稳定性界，在中心与局部DP模型下达到近极小极大最优率，并建立匹配的极小极大下界。
In a circular convolution model, we aim to infer on the density of a circular random variable using observations contaminated by an additive measurement error. We highlight the interplay of the two problems: optimal testing and quadratic functional estimation. Under general regularity assumptions, we determine an upper bound for the minimax risk of estimation for the quadratic functional. The upper bound consists of two terms, one that mimics a classical bias-variance trade-off and a second that causes the typical elbow effect in quadratic functional estimation. Using a minimax optimal estimat…
This paper presents minimax rates for density estimation when the data dimension $d$ is allowed to grow with the number of observations $n$ rather than remaining fixed as in previous analyses. We prove a non-asymptotic lower bound which gives the worst-case rate over standard classes of smooth densities, and we show that kernel density estimators achieve this rate. We also give oracle choices for the bandwidth and derive the fastest rate $d$ can grow with $n$ to maintain estimation consistency.
In this paper, we introduce a new estimator for the emission densities of a nonparametric hidden Markov model. It is adaptive and minimax with respect to each state's regularity--as opposed to globally minimax estimators, which adapt to the worst regularity among the emission densities. Our method is based on Goldenshluger and Lepski's methodology. It is computationally efficient and only requires a family of preliminary estimators, without any restriction on the type of estimators considered. We present two such estimators that allow to reach minimax rates up to a logarithmic term: a spectral…
We consider the problem of estimating the predictive density of future observations from a non-parametric regression model. The density estimators are evaluated under Kullback--Leibler divergence and our focus is on establishing the exact asymptotics of minimax risk in the case of Gaussian errors. We derive the convergence rate and constant for minimax risk among Bayesian predictive densities under Gaussian priors and we show that this minimax risk is asymptotically equivalent to that among all density estimators.
We propose a simple recursive data-based partitioning scheme which produces piecewise-constant or piecewise-linear density estimates on intervals, and show how this scheme can determine the optimal $L_1$ minimax rate for some discrete nonparametric classes.
We provide a minimax optimal estimation procedure for F and W in matrix valued linear models Y = F W + Z where the parameter matrix W and the design matrix F are unknown but the latter takes values in a known finite set. The proposed finite alphabet linear model is justified in a variety of applications, ranging from signal processing to cancer genetics. We show that this allows to separate F and W uniquely under weak identifiability conditions, a task which is not doable, in general. To this end we quantify in the noiseless case, that is, Z = 0, the perturbation range of Y in order to obtain …
We address the problem of adaptive minimax estimation in white gaussian noise model under $L_p$--loss, $1\leq p\leq\infty,$ on the anisotropic Nikolskii classes. We present the estimation procedure based on a new data-driven selection scheme from the family of kernel estimators with varying bandwidths. For proposed estimator we establish so-called Lp-norm oracle inequality and use it for deriving minimax adaptive results. We prove the existence of rate-adaptive estimators and fully characterize behavior of the minimax risk for different relationships between regularity parameters and norm inde…
Minimax optimal density estimation using a shallow generative model with a one-dimensional latent variable
本文从非参数密度估计角度研究VAE类隐式密度估计器的统计性质，证明在真实密度属于局部Hölder类时，一维潜变量浅层生成模型能达到Hellinger度量下的近最小最大最优收敛速率。
This article studies nonparametric methods to estimate the co-integrated volatility for multi-dimensional Lévy processes with high frequency data. We construct a spectral estimator for the co-integrated volatility and prove minimax rates for an appropriate bounded nonparametric class of semimartingales. Given $ n $ observations of increments over intervals of length $1/n$, the rates of convergence are $1 / \sqrt{n} $ if $ r \leq 1 $ and $ (n\log n)^{(r-2)/2} $ if $ r>1 $, which are optimal in a minimax sense. We bound the co-jump index activity from below with the harmonic mean. Finally, we as…
We consider estimating the predictive density under Kullback-Leibler loss in an $\ell_0$ sparse Gaussian sequence model. Explicit expressions of the first order minimax risk along with its exact constant, asymptotically least favorable priors and optimal predictive density estimates are derived. Compared to the sparse recovery results involving point estimation of the normal mean, new decision theoretic phenomena are seen. Suboptimal performance of the class of plug-in density estimates reflects the predictive nature of the problem and optimal strategies need diversification of the future risk…
This paper studies the minimax rate of nonparametric conditional density estimation under a weighted absolute value loss function in a multivariate setting. We first demonstrate that conditional density estimation is impossible if one only requires that $p_{X|Z}$ is smooth in $x$ for all values of $z$. This motivates us to consider a sub-class of absolutely continuous distributions, restricting the conditional density $p_{X|Z}(x|z)$ to not only be Hölder smooth in $x$, but also be total variation smooth in $z$. We propose a corresponding kernel-based estimator and prove that it achieves the mi…
We address the problem of density estimation with $\mathbb{L}_s$-loss by selection of kernel estimators. We develop a selection procedure and derive corresponding $\mathbb{L}_s$-risk oracle inequalities. It is shown that the proposed selection rule leads to the estimator being minimax adaptive over a scale of the anisotropic Nikol'skii classes. The main technical tools used in our derivations are uniform bounds on the $\mathbb{L}_s$-norms of empirical processes developed recently by Goldenshluger and Lepski [Ann. Probab. (2011), to appear].
For the important classical problem of inference on a sparse high-dimensional normal mean vector, we propose a novel empirical Bayes model that admits a posterior distribution with desirable properties under mild conditions. In particular, our empirical Bayes posterior distribution concentrates on balls, centered at the true mean vector, with squared radius proportional to the minimax rate, and its posterior mean is an asymptotically minimax estimator. We also show that, asymptotically, the support of our empirical Bayes posterior has roughly the same effective dimension as the true sparse mea…
Nearly-minimax variance estimation under rough random design
研究詹姆斯·罗宾斯关于随机设计下非参数回归估计常数条件方差问题，通过特定条件设定，证明极小极大均方根风险以\(n^{-\beta}\)为下界，否定了推测速率\(n^{-4s/(d + 4s)}\)的一致可达到性。
This paper characterizes the minimax linear estimator of the value of an unknown function at a boundary point of its domain in a Gaussian white noise model under the restriction that the first-order derivative of the unknown function is Lipschitz continuous (the second-order Hölder class). The result is then applied to construct the minimax optimal estimator for the regression discontinuity design model, where the parameter of interest involves function values at boundary points.
Minimax estimation of functionals in sparse vector model with correlated observations
本文研究了在部分关于协方差矩阵信息的情况下，稀疏向量的l2范数估计和零假设检验的极小极大最优方法，发现Frobenius范数决定了估计和检验的极小极大速率。
Minimax and adaptive estimation of general linear functionals under sparsity
研究高维稀疏均值向量下线性函数L(θ)=η^Tθ的非渐近最小最大估计，提出结合插值估计和阈值处理的估计器，通过加载依赖稀疏先验得到匹配下界，扩展至非对称噪声和未知噪声方差情况，展示异质性对最小最大和自适应速率的影响。
We consider the estimation of the global mode of a density under some decay rate condition around the global mode. We show that the maximum of a histogram, with proper choice of bandwidth, achieves the minimax rate that we establish for the setting that we consider. This is based on knowledge of the decay rate. Addressing the situation where the decay rate is unknown, we propose a multiscale variant consisting in the recursive refinement of a histogram, which is shown to be minimax adaptive. These methods run in linear time, and we prove in an appendix that this is best possible: There is no e…
Minimax rates of convergence for the nonparametric estimation of the diffusion coefficient from time-homogeneous SDE paths
本文针对时间齐次SDE扩散过程，在高频离散观测下研究扩散系数平方的非参数估计，建立了不同观测方案下Hölder空间中的极小极大收敛速率。
We study minimax density estimation on the product space $\mathbb{R}^{d_1}\times\mathbb{R}^{d_2}$. We consider $L^p$-risk for probability density functions defined over regularity spaces that allow for different level of smoothness in each of the variables. Precisely, we study probabilities on Sobolev spaces with dominating mixed-smoothness. We provide the rate of convergence that is optimal even for the classical Sobolev spaces.
For the Gaussian sequence model, we obtain non-asymptotic minimax rates of estimation of the linear, quadratic and the L2-norm functionals on classes of sparse vectors and construct optimal estimators that attain these rates. The main object of interest is the class s-sparse vectors for which we also provide completely adaptive estimators (independent of s and of the noise variance) having only logarithmically slower rates than the minimax ones. Furthermore, we obtain the minimax rates on the Lq-balls where 0 < q < 2. This analysis shows that there are, in general, three zones in the rates of …
The space of positive definite symmetric matrices has been studied extensively as a means of understanding dependence in multivariate data along with the accompanying problems in statistical inference. Many books and papers have been written on this subject, and more recently there has been considerable interest in high-dimensional random matrices with particular emphasis on the distribution of certain eigenvalues. With the availability of modern data acquisition capabilities, smoothing or nonparametric techniques are required that go beyond those applicable only to data arising in Euclidean s…
Minimax Optimal Probability Matrix Estimation For Graphon With Spectral Decay
本文针对具有特征值谱衰减的图振子，研究随机图概率矩阵的极小极大最优估计，证明谱阈值算法可达与信息论下界仅差对数因子的速率，并揭示其不同于光滑图振子设定的计算—统计含义。
In this paper, we study the minimax estimation of the Bochner integral $$μ_k(P):=\int_{\mathcal{X}} k(\cdot,x)\,dP(x),$$ also called as the kernel mean embedding, based on random samples drawn i.i.d.~from $P$, where $k:\mathcal{X}\times\mathcal{X}\rightarrow\mathbb{R}$ is a positive definite kernel. Various estimators (including the empirical estimator), $\hatθ_n$ of $μ_k(P)$ are studied in the literature wherein all of them satisfy $\bigl\| \hatθ_n-μ_k(P)\bigr\|_{\mathcal{H}_k}=O_P(n^{-1/2})$ with $\mathcal{H}_k$ being the reproducing kernel Hilbert space induced by $k$. The main contribution…
Statistical inference from high-dimensional data with low-dimensional structures has recently attracted lots of attention. In machine learning, deep generative modeling approaches implicitly estimate distributions of complex objects by creating new samples from the underlying distribution, and have achieved great success in generating synthetic realistic-looking images and texts. A key step in these approaches is the extraction of latent features or representations (encoding) that can be used for accurately reconstructing the original data (decoding). In other words, low-dimensional manifold s…
Canonical correlation analysis is a widely used multivariate statistical technique for exploring the relation between two sets of variables. This paper considers the problem of estimating the leading canonical correlation directions in high-dimensional settings. Recently, under the assumption that the leading canonical correlation directions are sparse, various procedures have been proposed for many high-dimensional applications involving massive data sets. However, there has been few theoretical justification available in the literature. In this paper, we establish rate-optimal nonasymptotic …
Asymptotic Minimax Estimation under Global-Local Priors
本文针对全局-局部先验，在放宽样本方差已知的假设、同时对误差方差赋予先验的情况下，推导收缩因子新尾界，给出目标参数后验均值的渐近极小极大速率。
Minimax Optimal Estimation of Mean and Covariance Functions with Spectral Regularization
本文研究了在再生核希尔伯特空间中利用频谱正则化估计均值和协方差函数的方法，推导了估计器的收敛率并建立了最优性界。
This paper surveys some recent developments in fundamental limits and optimal algorithms for network analysis. We focus on minimax optimal rates in three fundamental problems of network analysis: graphon estimation, community detection, and hypothesis testing. For each problem, we review state-of-the-art results in the literature followed by general principles behind the optimal procedures that lead to minimax estimation and testing. This allows us to connect problems in network analysis to other statistical inference problems from a general perspective.
Minimax Estimation of Distances on a Surface and Minimax Manifold Learning in the Isometric-to-Convex Setting
本研究针对光滑子流形内蕴距离估计问题，证明通过曲面重构可实现极小极大最优性并采用切向Delaunay复形构建网格，同时提出在重构曲面计算距离的Isomap变体，对等距流形学习问题具有极小极大最优性。
This paper considers the partially functional linear model (PFLM) where all predictive features consist of a functional covariate and a high dimensional scalar vector. Over an infinite dimensional reproducing kernel Hilbert space, the proposed estimation for PFLM is a least square approach with two mixed regularizations of a function-norm and an $\ell_1$-norm. Our main task in this paper is to establish the minimax rates for PFLM under high dimensional setting, and the optimal minimax rates of estimation is established by using various techniques in empirical process theory for analyzing kerne…
Wasserstein-based Minimax Estimation of Dependence in Multivariate Regularly Varying Extremes
本文首次为多元线性因子模型中谱测度估计建立极小极大风险界，以Wasserstein距离导出非渐近收敛速率并证明最优性，提出两步估计量达到下界，弥合极值理论中偏差-方差权衡的空白。
In a sparse high-dimensional elliptical model we consider a hard threshold estimator for the correlation matrix based on Kendall's tau with threshold level $α(\frac{\log p}{n})^{1/2}$. Parameters $α$ are identified such that the threshold estimator achieves the minimax rate under the squared Frobenius norm and the squared spectral norm. This allows a reasonable calibration of the estimator without any quantitative information about the tails of the underlying distribution. For Gaussian observations we even establish a critical threshold constant $α^\ast$ under the squared Frobenius norm, i.e. …
Minimax Rates for Conditional Density Estimation via Empirical Entropy
针对一致熵无法刻画条件密度估计极小极大速率的问题，研究以经验Hellinger熵为基础，在模型设定正确的情况下得到了极小极大KL风险的匹配上下界，仅需条件密度有上界即可。
Minimax rates for latent position estimation in the generalized random dot product graph
本文在二至无穷范数下建立了RDPG和GRDPG模型中潜位置估计的极小极大下界，证明特定谱嵌入方法可达到该下界，并利用Hadamard矩阵构造证明了低秩加噪声网络模型子空间估计的对应下界。
Resolvable Fourier Degree and Minimax Estimation on theBoolean Hypercube
本文研究布尔超立方体上傅里叶-索博列夫预算下的高维非参数回归，提出可解傅里叶度刻画极小极大风险，并扩展至单调回归及未知支撑情形。
The present paper considers a problem of estimating a linear functional $Φ=\int_{-\infty}^\infty φ(x) f(x)dx$ of an unknown deconvolution density $f$ on the basis of i.i.d. observations $Y_i = θ_i + ξ_i$ where $ξ_i$ has a known pdf $g$ and $f$ is the pdf of $θ_i$. Although various aspects and particular cases of this problem have been treated by a number of authors, there are still many gaps. In particular, there are no minimax lower bounds for an estimator of $Φ$ for an arbitrary function $φ$. The general upper risk bounds cover only the case when the Fourier transform of $φ$ exists. Moreover…
Driven by a wide range of applications, many principal subspace estimation problems have been studied individually under different structural constraints. This paper presents a unified framework for the statistical analysis of a general structured principal subspace estimation problem which includes as special cases non-negative PCA/SVD, sparse PCA/SVD, subspace constrained PCA/SVD, and spectral clustering. General minimax lower and upper bounds are established to characterize the interplay between the information-geometric complexity of the structural set for the principal subspaces, the sign…
In nonparametric statistics an optimality criterion for estimation procedures is provided by the minimax rate of convergence. However this classical point of view is subject to controversy as it requires to look for the worst behaviour reached by an estimation procedure in a given space. The purpose of this paper is to show that this is not justified as the minimax risk often coincides with a generic one. We are here interested in the rate of convergence attained by some classical estimators on almost every, in the sense of prevalence, function in a Besov space.
Nonparametric hazard rate estimation with associated kernels and minimax bandwidth choice
本文引入非参数危险率估计通用框架，用相关核并建立渐近结果，证明带宽选择不等式，提出构建分析新核的方法，还表明该框架适用于Gamma核并给出模拟与实验数据应用示例。
We consider model selection in generalized linear models (GLM) for high-dimensional data and propose a wide class of model selection criteria based on penalized maximum likelihood with a complexity penalty on the model size. We derive a general nonasymptotic upper bound for the expected Kullback-Leibler divergence between the true distribution of the data and that generated by a selected model, and establish the corresponding minimax lower bounds for sparse GLM. For the properly chosen (nonlinear) penalty, the resulting penalized maximum likelihood estimator is shown to be asymptotically minim…
We consider the problem of estimating the tail index $α$ of a distribution satisfying a $(α, β)$ second-order Pareto-type condition, where βis the second-order coefficient. When $β$ is available, it was previously proved that $α$ can be estimated with the oracle rate $n^{-β/(2β+1)}$. On the contrary, when $β$ is not available, estimating $α$ with the oracle rate is challenging; so additional assumptions that imply the estimability of $β$ are usually made. In this paper, we propose an adaptive estimator of $α$, and show that this estimator attains the rate $(n/\log\log n)^{-β/(2β+1)}$ without a…
Minimax Rates of Estimation for Optimal Transport Map between Infinite-Dimensional Spaces
本文研究了在无限维空间之间最优运输映射的估计，通过γ-光滑性概念证明了多项式率的最小最大最优率，并提出了一种估计方法用于函数数据分析。
Given a random sample of points from some unknown density, we propose a data-driven method for estimating density level sets under the r-convexity assumption. This shape condition generalizes the convexity property. However, the main problem in practice is that r is an unknown geometric characteristic of the set related to its curvature. A stochastic algorithm is proposed for selecting its optimal value from the data. The resulting reconstruction of the level set is able to achieve minimax rates for Hausdorff metric and distance in measure, up to log factors, uniformly on the level of the set.
Proper Bayes minimax multiple shrinkage estimation
研究在平方误差损失下估计多元正态均值问题，通过引入新方法证明适当贝叶斯极小极大多重收缩估计量存在，其能自适应向更有前景目标收缩，解决多目标选择挑战，且构建方式与以往不同。
We find the minimax rate of convergence in Hausdorff distance for estimating a manifold M of dimension d embedded in R^D given a noisy sample from the manifold. We assume that the manifold satisfies a smoothness condition and that the noise distribution has compact support. We show that the optimal rate of convergence is n^{-2/(2+d)}. Thus, the minimax rate depends only on the dimension of the manifold, not on the dimension of the space in which M is embedded.
Consider the problem of estimating the Shannon entropy of a distribution over $k$ elements from $n$ independent samples. We show that the minimax mean-square error is within universal multiplicative constant factors of $$\Big(\frac{k }{n \log k}\Big)^2 + \frac{\log^2 k}{n}$$ if $n$ exceeds a constant factor of $\frac{k}{\log k}$; otherwise there exists no consistent estimator. This refines the recent result of Valiant-Valiant \cite{VV11} that the minimal sample size for consistent entropy estimation scales according to $Θ(\frac{k}{\log k})$. The apparatus of best polynomial approximation plays…
This paper addresses an estimation problem of an additive functional of $ϕ$, which is defined as $θ(P;ϕ)=\sum_{i=1}^kϕ(p_i)$, given $n$ i.i.d. random samples drawn from a discrete distribution $P=(p_1,...,p_k)$ with alphabet size $k$. We have revealed in the previous paper that the minimax optimal rate of this problem is characterized by the divergence speed of the fourth derivative of $ϕ$ in a range of fast divergence speed. In this paper, we prove this fact for a more general range of the divergence speed. As a result, we show the minimax optimal rate of the additive functional estimation fo…
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。