首页/论文/1911.08085

arXiv 1911.080852019-11-19cs.DScs.LGstat.ML

Outlier-Robust High-Dimensional Sparse Estimation via Iterative Filtering

University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；UT Austin（德克萨斯大学奥斯汀分校）；University of California, San Diego（加利福尼亚大学圣迭戈分校）；Web3 Foundation（Web3基金会）

Ilias Diakonikolas, Sushrut Karmalkar, Daniel Kane, Eric Price, Alistair Stewart

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

We study high-dimensional sparse estimation tasks in a robust setting where a constant fraction of the dataset is adversarially corrupted. Specifically, we focus on the fundamental problems of robust sparse mean estimation and robust sparse PCA. We give the first practically viable robust estimators for these problems. In more detail, our algorithms are sample and computationally efficient and achieve near-optimal robustness guarantees. In contrast to prior provable algorithms which relied on the ellipsoid method, our algorithms use spectral techniques to iteratively remove outliers from the dataset. Our experimental evaluation on synthetic data shows that our algorithms are scalable and significantly outperform a range of previous approaches, nearly matching the best error rate without corruptions.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Structured and Unstructured Outlier Identification for Robust PCA: A Non iterative, Parameter free Algorithm

Indian Institute of Technology Madras（马德拉斯印度理工学院）

Robust PCA, the problem of PCA in the presence of outliers has been extensively investigated in the last few years. Here we focus on Robust PCA in the outlier model where each column of the data matrix is either an inlier or an outlier. Most of the existing methods for this model assumes either the knowledge of the dimension of the lower dimensional subspace or the fraction of outliers in the system. However in many applications knowledge of these parameters is not available. Motivated by this we propose a parameter free outlier identification method for robust PCA which a) does not require th…

2018-09-11cs.LGstat.ML

Identifying Outliers in Large Matrices via Randomized Adaptive Compressive Sampling

University of Minnesota – Twin Cities（明尼苏达大学双城分校）

This paper examines the problem of locating outlier columns in a large, otherwise low-rank, matrix. We propose a simple two-step adaptive sensing and inference approach and establish theoretical guarantees for its performance; our results show that accurate outlier identification is achievable using very few linear summaries of the original data matrix -- as few as the squared rank of the low-rank component plus the number of outliers, times constant and logarithmic factors. We demonstrate the performance of our approach experimentally in two stylized applications, one motivated by robust coll…

2014-11-19cs.ITcs.LGmath.IT

Self-Paced Probabilistic Principal Component Analysis for Data with Outliers

Tsinghua University（清华大学）；Peng Cheng Laboratory（鹏城实验室）

Principal Component Analysis (PCA) is a popular tool for dimensionality reduction and feature extraction in data analysis. There is a probabilistic version of PCA, known as Probabilistic PCA (PPCA). However, standard PCA and PPCA are not robust, as they are sensitive to outliers. To alleviate this problem, this paper introduces the Self-Paced Learning mechanism into PPCA, and proposes a novel method called Self-Paced Probabilistic Principal Component Analysis (SP-PPCA). Furthermore, we design the corresponding optimization algorithm based on the alternative search strategy and the expectation-…

2019-04-13cs.LGstat.ML

一种快速迭代鲁棒主成分分析方法

A Fast Iterative Robust Principal Component Analysis Method

提出快速迭代鲁棒PCA方法，通过增量PCA迭代估计内点位置和协方差，减轻离群值影响，在模拟和真实数据上验证了竞争性精度与更强鲁棒性。

2025-06-19cs.CEmath.STstat.TH

Outlier Detection for Text Data : An Extended Version

Oak Ridge National Laboratory（橡树岭国家实验室）；Korea University of Technology and Education（韩国技术教育大学）；IBM T. J. Watson Research Center（IBM T. J. 沃森研究中心）；Georgia Institute of Technology（佐治亚理工学院）

The problem of outlier detection is extremely challenging in many domains such as text, in which the attribute values are typically non-negative, and most values are zero. In such cases, it often becomes difficult to separate the outliers from the natural variations in the patterns in the underlying data. In this paper, we present a matrix factorization method, which is naturally able to distinguish the anomalies with the use of low rank approximations of the underlying data. Our iterative algorithm TONMF is based on block coordinate descent (BCD) framework. We define blocks over the term-docu…

2017-01-05cs.IRcs.LGstat.ML

Thresholding based Efficient Outlier Robust PCA

Microsoft Research India（微软研究院印度分院）

We consider the problem of outlier robust PCA (OR-PCA) where the goal is to recover principal directions despite the presence of outlier data points. That is, given a data matrix $M^*$, where $(1-α)$ fraction of the points are noisy samples from a low-dimensional subspace while $α$ fraction of the points can be arbitrary outliers, the goal is to recover the subspace accurately. Existing results for \OR-PCA have serious drawbacks: while some results are quite weak in the presence of noise, other results have runtime quadratic in dimension, rendering them impractical for large scale applications…

2017-02-18cs.LG

更多相关论文收起更多论文

Partial estimators and application to covariance estimation of gaussian and elliptical distributions

Robustness to outliers is often a desirable property of statistical estimators. Indeed many well known estimators offer very good optimal performance in theory but are unusable in applied contexts because of their sensitivity to outliers. Of particular interest to the authors is the case of covariance estimators in adaptive matched filtering schemes in signal processing applications such as RADAR and SONAR detection, for which a contamination by outliers of the estimated noise covariance can lead to a great impact on performances, in particular when these outliers are similar to the target sig…

2016-11-30math.STstat.TH

通过案例加权和单元加权实现稳健主成分

Robust Principal Components by Casewise and Cellwise Weighting

提出 cellPCA 方法，通过结合两种稳健损失函数和迭代重加权最小二乘算法，同时处理案例异常值、单元异常值和缺失数据，实现稳健的主成分分析。

2025-10-31stat.COstat.ME

Constant Approximation for $k$-Median and $k$-Means with Outliers via Iterative Rounding

In this paper, we present a new iterative rounding framework for many clustering problems. Using this, we obtain an $(α_1 + ε\leq 7.081 + ε)$-approximation algorithm for $k$-median with outliers, greatly improving upon the large implicit constant approximation ratio of Chen [Chen, SODA 2018]. For $k$-means with outliers, we give an $(α_2+ε\leq 53.002 + ε)$-approximation, which is the first $O(1)$-approximation for this problem. The iterative algorithm framework is very versatile; we show how it can be used to give $α_1$- and $(α_1 + ε)$-approximation algorithms for matroid and knapsack median …

2018-04-06cs.DS

Structural Iterative Rounding for Generalized $k$-Median Problems

This paper considers approximation algorithms for generalized $k$-median problems. This class of problems can be informally described as $k$-median with a constant number of extra constraints, and includes $k$-median with outliers, and knapsack median. Our first contribution is a pseudo-approximation algorithm for generalized $k$-median that outputs a $6.387$-approximate solution, with a constant number of fractional variables. The algorithm builds on the iterative rounding framework introduced by Krishnaswamy, Li, and Sandeep for $k$-median with outliers. The main technical innovation is allo…

2020-09-02cs.DS

List-Decodable Mean Estimation via Iterative Multi-Filtering

University of Wisconsin, Madison（威斯康星大学麦迪逊分校）；University of California, San Diego（加利福尼亚大学圣迭戈分校）

We study the problem of {\em list-decodable mean estimation} for bounded covariance distributions. Specifically, we are given a set $T$ of points in $\mathbb{R}^d$ with the promise that an unknown $α$-fraction of points in $T$, where $0< α< 1/2$, are drawn from an unknown mean and bounded covariance distribution $D$, and no assumptions are made on the remaining points. The goal is to output a small list of hypothesis vectors such that at least one of them is close to the mean of $D$. We give the first practically viable estimator for this problem. In more detail, our algorithm is sample and co…

2020-06-20cs.DScs.LGmath.ST

Outlier Robust Mean Estimation with Subgaussian Rates via Stability

University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of California, San Diego（加利福尼亚大学圣迭戈分校）

We study the problem of outlier robust high-dimensional mean estimation under a finite covariance assumption, and more broadly under finite low-degree moment assumptions. We consider a standard stability condition from the recent robust statistics literature and prove that, except with exponentially small failure probability, there exists a large fraction of the inliers satisfying this condition. As a corollary, it follows that a number of recently developed algorithms for robust mean estimation, including iterative filtering and non-convex gradient descent, give optimal error estimators with …

2021-03-16cs.DScs.LGmath.ST

迭代过滤在有监督学习中的强大作用（含重污染）

The Power of Iterative Filtering for Supervised Learning with (Heavy) Contamination

UT Austin（德克萨斯大学奥斯汀分校）

本文提出迭代多项式过滤算法，解决了有污染的有监督学习中的复杂性差距问题，首次实现了在高斯分布下高效学习半空间，以及在重污染下近最优的学习保证。

2026-01-12cs.DScs.LGstat.ML

迭代 Chow 过滤用于分布偏移学习

Iterative Chow Filtering for Learning with Distribution Shift

UT Austin（德克萨斯大学奥斯汀分校）

本文提出了一种基于迭代 Chow 过滤的方法，解决了分布偏移学习中的效率问题，展示了在 DNF 公式下实现准多项式时间 PQ 学习算法，并在多个函数类上提供了指数级改进。

2026-05-17cs.DScs.LG

快速各向同性中值滤波

Fast Isotropic Median Filtering

Google Research（谷歌研究）

针对中值滤波受限于位深、核大小及核形状且方核易产生伪影的问题，首次提出一种高效处理任意位深、任意核大小及任意凸核形状（含圆形）的方法。

2025-05-28cs.CVcs.DS

Layered Sampling for Robust Optimization Problems

In real world, our datasets often contain outliers. Moreover, the outliers can seriously affect the final machine learning result. Most existing algorithms for handling outliers take high time complexities (e.g. quadratic or cubic complexity). {\em Coreset} is a popular approach for compressing data so as to speed up the optimization algorithms. However, the current coreset methods cannot be easily extended to handle the case with outliers. In this paper, we propose a new variant of coreset technique, {\em layered sampling}, to deal with two fundamental robust optimization problems: {\em $k$-m…

2020-02-27cs.CGcs.DSmath.OC