首页/论文/2004.04348

arXiv 2004.043482020-04cs.ITmath.FAmath.IT

On the sparsity of LASSO minimizers in sparse data recovery

Simon Foucart, Eitan Tadmor, Ming Zhong

英文摘要：

We present a detailed analysis of the unconstrained $\ell_1$-weighted LASSO method for recovery of sparse data from its observation by randomly generated matrices, satisfying the Restricted Isometry Property (RIP) with constant $δ<1$, and subject to negligible measurement and compressibility errors. We prove that if the data is $k$-sparse, then the size of support of the LASSO minimizer, $s$, maintains a comparable sparsity, $s\leq C_δk$. For example, if $δ=0.7$ then $s< 11k$ and a slightly smaller $δ=0.4$ yields $s< 4k$. We also derive new $\ell_2/\ell_1$ error bounds which highlight precise dependence on $k$ and on the LASSO parameter $λ$, before the error is driven below the scale of negligible measurement/ and compressiblity errors.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

The Sparsity of LASSO-type Minimizers

This note extends an attribute of the LASSO procedure to a whole class of related procedures, including square-root LASSO, square LASSO, LAD-LASSO, and an instance of generalized LASSO. Namely, under the assumption that the input matrix satisfies an $\ell_p$-restricted isometry property (which in some sense is weaker than the standard $\ell_2$-restricted isometry property assumption), it is shown that if the input vector comes from the exact measurement of a sparse vector, then the minimizer of any such LASSO-type procedure has sparsity comparable to the sparsity of the measured vector. The re…

2022-09-12cs.ITmath.FAmath.IT

Robust Regression and Lasso

McGill University（麦吉尔大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）

Lasso, or $\ell^1$ regularized least squares, has been explored extensively for its remarkable sparsity properties. It is shown in this paper that the solution to Lasso, in addition to its sparsity, has robustness properties: it is the solution to a robust optimization problem. This has two important consequences. First, robustness provides a connection of the regularizer to a physical property, namely, protection from noise. This allows a principled selection of the regularizer, and in particular, generalizations of Lasso that also yield convex optimization problems are obtained by considerin…

2008-11-11cs.ITcs.LGmath.IT

Collaborative Hierarchical Sparse Modeling

Sparse modeling is a powerful framework for data analysis and processing. Traditionally, encoding in this framework is done by solving an l_1-regularized linear regression problem, usually called Lasso. In this work we first combine the sparsity-inducing property of the Lasso model, at the individual feature level, with the block-sparsity property of the group Lasso model, where sparse groups of features are jointly encoded, obtaining a sparsity pattern hierarchically structured. This results in the hierarchical Lasso, which shows important practical modeling advantages. We then extend this ap…

2010-03-01cs.ITmath.IT

Gradually Atom Pruning for Sparse Reconstruction and Extension to Correlated Sparsity

We propose a new algorithm for recovery of sparse signals from their compressively sensed samples. The proposed algorithm benefits from the strategy of gradual movement to estimate the positions of non-zero samples of sparse signal. We decompose each sample of signal into two variables, namely "value" and "detector", by a weighted exponential function. We update these new variables using gradient descent method. Like the traditional compressed sensing algorithms, the first variable is used to solve the Least Absolute Shrinkage and Selection Operator (Lasso) problem. As a new strategy, the seco…

2012-04-03cs.ITmath.IT

Necessary and Sufficient Conditions on Sparsity Pattern Recovery

The problem of detecting the sparsity pattern of a k-sparse vector in R^n from m random noisy measurements is of interest in many areas such as system identification, denoising, pattern recognition, and compressed sensing. This paper addresses the scaling of the number of measurements m, with signal dimension n and sparsity-level nonzeros k, for asymptotically-reliable detection. We show a necessary condition for perfect recovery at any given SNR for all algorithms, regardless of complexity, is m = Omega(k log(n-k)) measurements. Conversely, it is shown that this scaling of Omega(k log(n-k)) m…

2008-04-11cs.ITmath.IT

Multi-target Radar Detection within a Sparsity Framework

Traditional radar detection schemes are typically studied for single target scenarios and they can be non-optimal when there are multiple targets in the scene. In this paper, we develop a framework to discuss multi-target detection schemes with sparse reconstruction techniques that is based on the Neyman-Pearson criterion. We will describe an initial result in this framework concerning false alarm probability with LASSO as the sparse reconstruction technique. Then, several simulations validating this result will be discussed. Finally, we describe several research avenues to further pursue this…

2013-11-06cs.ITmath.IT

更多相关论文收起更多论文

Sharp thresholds for high-dimensional and noisy recovery of sparsity

The problem of consistently estimating the sparsity pattern of a vector $\betastar \in \real^\mdim$ based on observations contaminated by noise arises in various contexts, including subset selection in regression, structure estimation in graphical models, sparse approximation, and signal denoising. We analyze the behavior of $\ell_1$-constrained quadratic programming (QP), also referred to as the Lasso, for recovering the sparsity pattern. Our main result is to establish a sharp relation between the problem dimension $\mdim$, the number $\spindex$ of non-zero elements in $\betastar$, and the n…

2006-05-30cs.ITmath.ITmath.ST

Iterative Adaptively Regularized LASSO-ADMM Algorithm for CFAR Estimation of Sparse Signals: IAR-LASSO-ADMM-CFAR Algorithm

The least-absolute shrinkage and selection operator (LASSO) is a regularization technique for estimating sparse signals of interest emerging in various applications and can be efficiently solved via the alternating direction method of multipliers (ADMM), which will be termed as LASSO-ADMM algorithm. The choice of the regularization parameter has significant impact on the performance of LASSO-ADMM algorithm. However, the optimization for the regularization parameter in the existing LASSO-ADMM algorithms has not been solved yet. In order to optimize this regularization parameter, we propose an e…

2022-08-24cs.ITeess.SPmath.IT

通过L1最小化从含噪随机测量中实现精确支持恢复

Sharp Support Recovery from Noisy Random Measurements by L1 minimization

Université Bordeaux 1（波尔多大学）；Université Paris-Dauphine（巴黎第九大学）；Université Caen（卡昂大学）

本文研究在测量矩阵为高斯随机矩阵时，通过L1惩罚最小化（Lasso）从含噪测量中恢复信号支撑集的精确非渐近界，包括严格稀疏和近似稀疏情形，并给出显式锐利常数。

2011-09-12cs.ITcs.NAmath.IT

Compressed Sensing for Block-Sparse Smooth Signals

We present reconstruction algorithms for smooth signals with block sparsity from their compressed measurements. We tackle the issue of varying group size via group-sparse least absolute shrinkage selection operator (LASSO) as well as via latent group LASSO regularizations. We achieve smoothness in the signal via fusion. We develop low-complexity solvers for our proposed formulations through the alternating direction method of multipliers.

2013-09-10cs.ITmath.ITmath.ST

On the robustness of minimum norm interpolators and regularized empirical risk minimizers

This article develops a general theory for minimum norm interpolating estimators and regularized empirical risk minimizers (RERM) in linear models in the presence of additive, potentially adversarial, errors. In particular, no conditions on the errors are imposed. A quantitative bound for the prediction error is given, relating it to the Rademacher complexity of the covariates, the norm of the minimum norm interpolator of the errors and the size of the subdifferential around the true parameter. The general theory is illustrated for Gaussian features and several norms: The $\ell_1$, $\ell_2$, g…

2021-10-07cs.ITcs.NAmath.IT

False Discoveries Occur Early on the Lasso Path

In regression settings where explanatory variables have very low correlations and there are relatively few effects, each of large magnitude, we expect the Lasso to find the important variables with few errors, if any. This paper shows that in a regime of linear sparsity---meaning that the fraction of variables with a non-vanishing effect tends to a constant, however small---this cannot really be the case, even when the design variables are stochastically independent. We demonstrate that true features and null features are always interspersed on the Lasso path, and that this phenomenon occurs n…

2016-09-15cs.ITmath.ITmath.ST