The purpose of this paper is to study the problem of estimating a compactly supported density of probability from noisy observations of its moments. In fact, we provide a statistical approach to the famous Hausdorff classical moment problem. We prove an upper bound and a lower bound on the rate of convergence of the mean squared error showing that the considered estimator attains minimax rate over the corresponding smoothness classes.
Given a random sample of points from some unknown density, we propose a data-driven method for estimating density level sets under the r-convexity assumption. This shape condition generalizes the convexity property. However, the main problem in practice is that r is an unknown geometric characteristic of the set related to its curvature. A stochastic algorithm is proposed for selecting its optimal value from the data. The resulting reconstruction of the level set is able to achieve minimax rates for Hausdorff metric and distance in measure, up to log factors, uniformly on the level of the set.
We derive non-asymptotic minimax bounds for the Hausdorff estimation of $d$-dimensional submanifolds $M \subset \mathbb{R}^D$ with (possibly) non-empty boundary $\partial M$. The model reunites and extends the most prevalent $\mathcal{C}^2$-type set estimation models: manifolds without boundary, and full-dimensional domains. We consider both the estimation of the manifold $M$ itself and that of its boundary $\partial M$ if non-empty. Given $n$ samples, the minimax rates are of order $O\bigl((\log n/n)^{2/d}\bigr)$ if $\partial M = \emptyset$ and $O\bigl((\log n/n)^{2/(d+1)}\bigr)$ if $\partial…
We prove minimax bounds for estimating Gaussian location mixtures on $\mathbb{R}^d$ under the squared $L^2$ and the squared Hellinger loss functions. Under the squared $L^2$ loss, we prove that the minimax rate is upper and lower bounded by a constant multiple of $n^{-1}(\log n)^{d/2}$. Under the squared Hellinger loss, we consider two subclasses based on the behavior of the tails of the mixing measure. When the mixing measure has a sub-Gaussian tail, the minimax rate under the squared Hellinger loss is bounded from below by $(\log n)^{d}/n$. On the other hand, when the mixing measure is only …
我们得到的估计误差率的估计得到的正则化中位数的均值检验的聚合，建设乐凸轮。结果保持指数大的概率-在高斯框架与独立噪声-在数据上只有弱矩假设，而不假设噪声和设计之间的独立性。任何范数都可以用于正则化。当它有一定的稀疏诱导力，我们恢复稀疏收敛速度。该过程是稳健的，因为大部分数据可能已损坏，这些离群值与我们想要重建的预言机无关。我们的一般风险界限为\begin{equation*} \max\left(\mbox{minimax rate in the i.i.d. setup}, \frac{\text{number of outliers}}{\text{number of observations}}\right) \enspace. \end{equation*}阶。特别是，离群值的数量可能与（数据数量）$\times$（极小极大率）一样大，而不影响该率。其他数据不必同分布，但应仅具有相等的$L^1$和$L^2$矩。例如，当噪声对于一些$q_0>2$具有$q_0$矩时，通过LASSO的中值版本以指数大概率实现$\mathbb{R}^d$中的$s$稀疏向量的恢复的最小最大速率$s \log(ed/s)/N$，设计矩阵的条目应该具有$C_0\log(ed)$矩，并且数据集可以被破坏到$C_1 s \log(ed/s)$离群值。
A popular class of problem in statistics deals with estimating the support of a density from $n$ observations drawn at random from a $d$-dimensional distribution. The one-dimensional case reduces to estimating the end points of a univariate density. In practice, an experimenter may only have access to a noisy version of the original data. Therefore, a more realistic model allows for the observations to be contaminated with additive noise. In this paper, we consider estimation of convex bodies when the additive noise is distributed according to a multivariate Gaussian distribution, even though …
The Wasserstein metric is an important measure of distance between probability distributions, with applications in machine learning, statistics, probability theory, and data analysis. This paper provides upper and lower bounds on statistical minimax rates for the problem of estimating a probability distribution under Wasserstein loss, using only metric properties, such as covering and packing numbers, of the sample space, and weak moment assumptions on the probability distributions.
We study the minimax estimation of $α$-divergences between discrete distributions for integer $α\ge 1$, which include the Kullback--Leibler divergence and the $χ^2$-divergences as special examples. Dropping the usual theoretical tricks to acquire independence, we construct the first minimax rate-optimal estimator which does not require any Poissonization, sample splitting, or explicit construction of approximating polynomials. The estimator uses a hybrid approach which solves a problem-independent linear program based on moment matching in the non-smooth regime, and applies a problem-dependent…
A weighted Shiryaev-Roberts change detection procedure is shown to approximately minimize the expected delay to detection as well as higher moments of the detection delay among all change-point detection procedures with the given low maximal local probability of a false alarm within a window of a fixed length in pointwise and minimax settings for general non-i.i.d. data models and for the composite post-change hypothesis when the post-change parameter is unknown. We establish very general conditions for the models under which the weighted Shiryaev-Roberts procedure is asymptotically optimal. T…
Motivated by the growing popularity of variants of the Wasserstein distance in statistics and machine learning, we study statistical inference for the Sliced Wasserstein distance--an easily computable variant of the Wasserstein distance. Specifically, we construct confidence intervals for the Sliced Wasserstein distance which have finite-sample validity under no assumptions or under mild moment assumptions. These intervals are adaptive in length to the regularity of the underlying distributions. We also bound the minimax risk of estimating the Sliced Wasserstein distance, and as a consequence …
In this paper we study minimax and adaptation rates in general isotonic regression. For uniform deterministic and random designs in $[0,1]^d$ with $d\ge 2$ and $N(0,1)$ noise, the minimax rate for the $\ell_2$ risk is known to be bounded from below by $n^{-1/d}$ when the unknown mean function $f$ is nondecreasing and its range is bounded by a constant, while the least squares estimator (LSE) is known to nearly achieve the minimax rate up to a factor $(\log n)^γ$ where $n$ is sample size, $γ= 4$ in the lattice design and $γ= \max\{9/2, (d^2+d+1)/2 \}$ in the random design. Moreover, the LSE is …
Regularized least squares learning with heavy-tailed noise is minimax optimal
Merantix Momentum；Institute for Mathematical Stochastics, Technische Universität Braunschweig（数学概率研究所，布拉unsch维格技术大学）；Gatsby Computational Neuroscience Unit, University College London（盖茨计算神经科学单元，伦敦大学学院）
Online imitation learning is the problem of how best to mimic expert demonstrations, given access to the environment or an accurate simulator. Prior work has shown that in the infinite sample regime, exact moment matching achieves value equivalence to the expert policy. However, in the finite sample regime, even if one has no optimization error, empirical variance can lead to a performance gap that scales with $H^2 / N$ for behavioral cloning and $H / \sqrt{N}$ for online moment matching, where $H$ is the horizon and $N$ is the size of the expert dataset. We introduce the technique of replay e…
The conditional moment problem is a powerful formulation for describing structural causal parameters in terms of observables, a prominent example being instrumental variable regression. A standard approach reduces the problem to a finite set of marginal moment conditions and applies the optimally weighted generalized method of moments (OWGMM), but this requires we know a finite set of identifying moments, can still be inefficient even if identifying, or can be theoretically efficient but practically unwieldy if we use a growing sieve of moment conditions. Motivated by a variational minimax ref…
Optimal detection of general moment changes: Simultaneous mean and covariance change detection and beyond
University of Notre Dame（圣母大学）；University of California, San Diego（加利福尼亚大学圣地亚哥分校）；Auburn University（奥本大学）；Washington University in St. Louis（华盛顿大学圣路易斯分校）
We present \emph{Local Moment Matching (LMM)}, a unified methodology for symmetric functional estimation and distribution estimation under Wasserstein distance. We construct an efficiently computable estimator that achieves the minimax rates in estimating the distribution up to permutation, and show that the plug-in approach of our unlabeled distribution estimator is "universal" in estimating symmetric functionals of discrete distributions. Instead of doing best polynomial approximation explicitly as in existing literature of functional estimation, the plug-in approach conducts polynomial appr…
Tsinghua University（清华大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Many problems in causal inference and economics can be formulated in the framework of conditional moment models, which characterize the target function through a collection of conditional moment restrictions. For nonparametric conditional moment models, efficient estimation often relies on preimposed conditions on various measures of ill-posedness of the hypothesis space, which are hard to validate when flexible models are used. In this work, we address this issue by proposing a procedure that automatically learns representations with controlled measures of ill-posedness. Our method approximat…
University of Toronto（多伦多大学）；Canadian Institute for Advanced Research（加拿大高等研究院）
我们考虑从数据中学习深度生成模型的问题。我们制定了一种方法，该方法通过多层感知器的单次前馈来生成独立样本，如最近提出的生成对抗网络（Goodfellow et al.，2014年）。然而，训练一个生成对抗网络需要仔细优化一个困难的极小极大程序。相反，我们利用了一种被称为最大平均差异（MMD）的统计假设检验技术，这导致了一个简单的目标，可以被解释为匹配数据集和模型样本之间的所有统计阶数，并且可以通过反向传播进行训练。我们通过将我们的生成网络与自动编码器网络相结合，使用MMD来学习生成代码，然后可以解码生成样本，从而进一步提高了这种方法的性能。我们表明，这些技术的组合产生优秀的生成模型相比，MNIST和多伦多人脸数据库上测量的基线方法。
We develop an approach for estimating models described via conditional moment restrictions, with a prototypical application being non-parametric instrumental variable regression. We introduce a min-max criterion function, under which the estimation problem can be thought of as solving a zero-sum game between a modeler who is optimizing over the hypothesis space of the target model and an adversary who identifies violating moments over a test function space. We analyze the statistical estimation rate of the resulting estimator for arbitrary hypothesis spaces, with respect to an appropriate anal…
