An important part of the legacy of Evarist Giné is his fundamental contributions to our understanding of $U$-statistics and $U$-processes. In this paper we discuss the estimation of the mean of multivariate functions in case of possibly heavy-tailed distributions. In such situations, reliable estimates of the mean cannot be obtained by usual $U$-statistics. We introduce a new estimator, based on the so-called median-of-means technique. We develop performance bounds for this new estimator that generalizes an estimate of Arcones and Giné (1993), showing that the new estimator performs, under minimal moment conditions, as well as classical $U$-statistics for bounded random variables. We discuss an application of this estimator to clustering.
Consider the problem of simultaneous estimation of location and variance matrix under Huber's contaminated Gaussian model. First, we study minimum $f$-divergence estimation at the population level, corresponding to a generative adversarial method with a nonparametric discriminator and establish conditions on $f$-divergences which lead to robust estimation, similarly to robustness of minimum distance estimation. More importantly, we develop tractable adversarial algorithms with simple spline discriminators, which can be implemented via nested optimization such that the discriminator parameters …
Generative Adversarial Nets for Robust Scatter Estimation: A Proper Scoring Rule Perspective
University of Chicago（芝加哥大学）；Hong Kong University of Science and Technology（香港科技大学）
稳健散布估计是统计学中的一项基本任务。Gao et al.（2018）最近关于鲁棒估计和生成对抗网络（GANs）之间的联系的发现表明，可以使用优化GANs的类似技术来计算类似深度的鲁棒估计器。在本文中，我们介绍了一个一般的学习通过分类框架的基础上适当的评分规则的概念。这个框架允许我们通过$f$的变分近似的透镜来理解矩阵深度函数和各种GANs-由适当的评分规则引起的发散。然后，我们提出了一类新的强大的散布估计在这个框架中仔细构造鉴别器与适当的神经网络结构。在Huber污染模型下，证明了这些估计量能达到极小极大散布率估计.我们的数值结果表明，在各种设置下对文献中的竞争对手的良好性能。
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of California, San Diego（加利福尼亚大学圣迭戈分校）
We study the problem of outlier robust high-dimensional mean estimation under a finite covariance assumption, and more broadly under finite low-degree moment assumptions. We consider a standard stability condition from the recent robust statistics literature and prove that, except with exponentially small failure probability, there exists a large fraction of the inliers satisfying this condition. As a corollary, it follows that a number of recently developed algorithms for robust mean estimation, including iterative filtering and non-convex gradient descent, give optimal error estimators with …
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of Cambridge（剑桥大学）
Robust estimation is an important problem in statistics which aims at providing a reasonable estimator when the data-generating distribution lies within an appropriately defined ball around an uncontaminated distribution. Although minimax rates of estimation have been established in recent years, many existing robust estimators with provably optimal convergence rates are also computationally intractable. In this paper, we study several estimation problems under a Wasserstein contamination model and present computationally tractable estimators motivated by generative adversarial networks (GANs)…
Differential Privacy (DP) provides an elegant mathematical framework for defining a provable disclosure risk in the presence of arbitrary adversaries; it guarantees that whether an individual is in a database or not, the results of a DP procedure should be similar in terms of their probability distribution. While DP mechanisms are provably effective in protecting privacy, they often negatively impact the utility of the query responses, statistics and/or analyses that come as outputs from these mechanisms. To address this problem, we use ideas from the area of robust statistics which aims at re…
High-Dimensional Robust Mean Estimation via Gradient Descent
University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Duke University（杜克大学）；University of Southern California（南加州大学）
We study the problem of robust mean estimation and introduce a novel Hamming distance-based measure of distribution shift for coordinate-level corruptions. We show that this measure yields adversary models that capture more realistic corruptions than those used in prior works, and present an information-theoretic analysis of robust mean estimation in these settings. We show that for structured distributions, methods that leverage the structure yield information theoretically more accurate mean estimation. We also focus on practical algorithms for robust mean estimation and study when data clea…
In this paper we present robust estimators for one-shot device test data under lognormal lifetimes. Based on these estimators, confidence intervals and Wald-type tests are also developed. Their robustness feature is illustrated through a simulation study and two numerical examples.
University of California, San Diego（加利福尼亚大学圣地亚哥分校）
Real data are rarely pure. Hence the past half-century has seen great interest in robust estimation algorithms that perform well even when part of the data is corrupt. However, their vast majority approach optimal accuracy only when given a tight upper bound on the fraction of corrupt data. Such bounds are not available in practice, resulting in weak guarantees and often poor performance. This brief note abstracts the complex and pervasive robustness problem into a simple geometric puzzle. It then applies the puzzle's solution to derive a universal meta technique that converts any robust estim…
Adaptive smoothness of function estimation in the three classical problems of the non-parametrical statistic in the three classical problems of the non-parametrical statistic
The estimation of signal dimension under heavy-tailed latent factor models is studied. As a primary contribution, robust extensions of an earlier estimator based on Gaussian Stein's unbiased risk estimation are proposed. These novel extensions are based on the framework of elliptical distributions and robust scatter matrices. Extensive simulation studies are conducted in order to compare the novel methods with several well-known competitors in both estimation accuracy and computational speed. The novel methods are applied to a financial asset return data set.
Robust mixture regression modeling based on the Generalized M (GM)-estimation method
Bai（2010）and Bai et al.（2012）提出了基于M回归估计的稳健混合回归方法。然而，M-估计量对响应变量中的异常值是稳健的，但对解释变量中的异常值（杠杆点）不稳健。在本文中，我们提出了一个强大的混合回归过程来处理离群值和杠杆点，同时。我们提出的混合回归方法是基于GM回归估计。我们给出了一个期望最大化（EM）型算法来计算感兴趣的参数的估计。我们提供了一个模拟研究和一个真实的数据例子来评估所提出的方法对离群值和杠杆点的鲁棒性性能。
The main object of investigation in this paper is a very general regression model in optional setting - when an observed process is an optional semimartingale depending on an unknown parameter. It is well-known that statistical data may present an information flow/filtration without usual conditions. The estimation problem is achieved by means of structural least squares (LS) estimates and their sequential versions. The main results of the paper are devoted to the strong consistency of such LS-estimates. For sequential LS-estimates the property of fixed accuracy is proved.
In this paper, we introduce a robust nonparametric density estimator combining the popular Kernel Density Estimation method and the Median-of-Means principle (MoM-KDE). This estimator is shown to achieve robustness to any kind of anomalous data, even in the case of adversarial contamination. In particular, while previous works only prove consistency results under known contamination model, this work provides finite-sample high-probability error-bounds without a priori knowledge on the outliers. Finally, when compared with other robust kernel estimators, we show that MoM-KDE achieves competitiv…
We present an extension to the robust phase estimation protocol, which can identify incorrect results that would otherwise lie outside the expected statistical range. Robust phase estimation is increasingly a method of choice for applications such as estimating the effective process parameters of noisy hardware, but its robustness is dependent on the noise satisfying certain threshold assumptions. We provide consistency checks that can indicate when those thresholds have been violated, which can be difficult or impossible to test directly. We test these consistency checks for several common no…
We present a unifying view on various statistical estimation techniques including penalization, variational and thresholding methods. These estimators will be analyzed in the context of statistical linear inverse problems including nonparametric and change point regression, and high dimensional linear models as examples. Our approach reveals many seemingly unrelated estimation schemes as special instances of a general class of variational multiscale estimators, named MIND (MultIscale Nemirovskii--Dantzig). These estimators result from minimizing certain regularization functionals under convex …
Median-of-means (MOM) based procedures provide non-asymptotic and strong deviation bounds even when data are heavy-tailed and/or corrupted. This work proposes a new general way to bound the excess risk for MOM estimators. The core technique is the use of VC-dimension (instead of Rademacher complexity) to measure the statistical complexity. In particular, this allows to give the first robust estimators for sparse estimation which achieves the so-called subgaussian rate only assuming a finite second moment for the uncorrupted data. By comparison, previous works using Rademacher complexities requ…
Massachusetts Institute of Technology（麻省理工学院）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Coresets have emerged as a powerful tool to summarize data by selecting a small subset of the original observations while retaining most of its information. This approach has led to significant computational speedups but the performance of statistical procedures run on coresets is largely unexplored. In this work, we develop a statistical framework to study coresets and focus on the canonical task of nonparameteric density estimation. Our contributions are twofold. First, we establish the minimax rate of estimation achievable by coreset-based estimators. Second, we show that the practical core…
We propose a simple robust hypothesis test that has the same sample complexity as that of the optimal Neyman-Pearson test up to constants, but robust to distribution perturbations under Hellinger distance. We discuss the applicability of such a robust test for estimating distributions in Hellinger distance. We empirically demonstrate the power of the test on canonical distributions.
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We study the robust mean estimation problem in high dimensions, where $α<0.5$ fraction of the data points can be arbitrarily corrupted. Motivated by compressive sensing, we formulate the robust mean estimation problem as the minimization of the $\ell_0$-`norm' of the outlier indicator vector, under second moment constraints on the inlier data points. We prove that the global minimum of this objective is order optimal for the robust mean estimation problem, and we propose a general framework for minimizing the objective. We further leverage the $\ell_1$ and $\ell_p$ $(0<p<1)$, minimization tech…
