arXiv 1605.033352016-05-11stat.MEstat.ML

Asymptotic properties for combined $L_1$ and concave regularization

Yingying Fan, Jinchi Lv

备注：: 16 pages
发表信息：: Biometrika 101, 57-70
DOI：: 10.1093/biomet/ast047

英文摘要：

Two important goals of high-dimensional modeling are prediction and variable selection. In this article, we consider regularization with combined $L_1$ and concave penalties, and study the sampling properties of the global optimum of the suggested method in ultra-high dimensional settings. The $L_1$-penalty provides the minimum regularization needed for removing noise variables in order to achieve oracle prediction risk, while concave penalty imposes additional regularization to control model sparsity. In the linear model setting, we prove that the global optimum of our method enjoys the same oracle inequalities as the lasso estimator and admits an explicit bound on the false sign rate, which can be asymptotically vanishing. Moreover, we establish oracle risk inequalities for the method and the sampling properties of computable solutions. Numerical studies suggest that our method yields more stable estimates than using a concave penalty alone.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

A General Theory of Concave Regularization for High Dimensional Sparse Estimation Problems

Concave regularization methods provide natural procedures for sparse recovery. However, they are difficult to analyze in the high dimensional setting. Only recently a few sparse recovery results have been established for some specific local solutions obtained via specialized numerical procedures. Still, the fundamental relationship between these solutions such as whether they are identical or their relationship to the global minimizer of the underlying nonconvex formulation is unknown. The current paper fills this conceptual gap by presenting a general theoretical framework showing that under …

2012-02-11stat.ML

基于弱凸-凹正则化的稀疏回归中二次测量的支持恢复与ℓ₂误差界

Support Recovery and $\ell_2$-Error Bound for Sparse Regression with Quadratic Measurements via Weakly-Convex-Concave Regularization

University of Manitoba（曼尼托巴大学）

本文研究了高维二次测量模型中弱凸-凹正则化估计器的有限样本性质，通过弱凸-凹惩罚最小二乘方法，建立了局部极小值解的支持恢复和ℓ₂误差界，采用两种近端梯度策略求解优化问题。

2026-04-14cs.ITmath.ITmath.ST

Asymptotic equivalence of regularization methods in thresholded parameter space

High-dimensional data analysis has motivated a spectrum of regularization methods for variable selection and sparse modeling, with two popular classes of convex ones and concave ones. A long debate has been on whether one class dominates the other, an important question both in theory and to practitioners. In this paper, we characterize the asymptotic equivalence of regularization methods, with general penalty functions, in a thresholded parameter space under the generalized linear model setting, where the dimensionality can grow up to exponentially with the sample size. To assess their perfor…

2016-05-11math.STstat.MEstat.ML

稀疏输入神经网络使用群凹正则化

Sparse-Input Neural Network using Group Concave Regularization

School of Data Science and Analytics（数据科学与分析学院）；Kennesaw State University（肯尼斯州立大学）；Department of Biostatistics and Bioinformatics（生物统计学与生物信息学系）；Duke University（杜克大学）

本文提出了一种稀疏输入神经网络框架，利用群凹正则化进行特征选择，解决了高维数据中的变量选择和非线性估计问题。

2025-12-31cs.LGstat.ML

One-step estimator paths for concave regularization

The statistics literature of the past 15 years has established many favorable properties for sparse diminishing-bias regularization: techniques which can roughly be understood as providing estimation under penalty functions spanning the range of concavity between $L_0$ and $L_1$ norms. However, lasso $L_1$-regularized estimation remains the standard tool for industrial `Big Data' applications because of its minimal computational cost and the presence of easy-to-apply rules for penalty selection. In response, this article proposes a simple new algorithm framework that requires no more computati…

2016-05-01stat.AP

协整回归元：折叠凹惩罚超越LASSO的场景

Cointegrated Regressors: Where Folded Concave Penalties Go Beyond LASSO

Hunan University（湖南大学）；The Chinese University of Hong Kong（香港中文大学）；University of Macau（澳门大学）

本文针对高维协整回归中LASSO凸惩罚过度抑制系数的问题，采用折叠凹惩罚（SCAD、MCP），建立两类oracle性质，经模拟和欧元区失业率预测验证，其变量选择与样本外预测性能优于LASSO。

2026-10-19econ.EMstat.MEstat.ML

更多相关论文收起更多论文

非凸潜变量最优划分块稀疏恢复 via 对数和最小化凹惩罚

Nonconvex Latent Optimally Partitioned Block-Sparse Recovery via Log-Sum and Minimax Concave Penalties

Nagoya Institute of Technology（名古屋工业大学）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；Keio University（庆应义塾大学）

本文提出非凸正则化方法LogLOP-l2/l1和AdaLOP-l2/l1，用于恢复具有未知块划分的块稀疏信号，通过新颖的变分公式提升恢复精度，实验显示优于现有方法。

2026-03-01cs.LGstat.ML

Regularization for Cox's proportional hazards model with NP-dimensionality

High throughput genetic sequencing arrays with thousands of measurements per sample and a great amount of related censored clinical data have increased demanding need for better measurement specific model selection. In this paper we establish strong oracle properties of nonconcave penalized methods for nonpolynomial (NP) dimensional data with censoring in the framework of Cox's proportional hazards model. A class of folded-concave penalties are employed and both LASSO and SCAD are discussed specifically. We unveil the question under which dimensionality and correlation restrictions can an orac…

2012-05-25math.STstat.MEstat.ML

正则化与松弛：从锥优化视角看统计变量选择

Regularization vs. Relaxation: A conic optimization perspective of statistical variable selection

Department of Mathematics, Washington State University（数学系，华盛顿州立大学）；Department of Statistics, University of Connecticut（统计系，康涅狄格大学）；Department of Industrial and Systems Engineering, University of Wisconsin-Madison（工业与系统工程系，威斯康星大学麦迪逊分校）

本文从锥优化视角探讨变量选择问题，证明MCP和反Huber惩罚函数可视为视角松弛的特例，并通过半定松弛解决，结合Goemans-Williamson方法获得近似解。

2015-10-20cs.LGcs.NAmath.NA

High-Dimensional Quantile Regression: Convolution Smoothing and Concave Regularization

$\ell_1$-penalized quantile regression is widely used for analyzing high-dimensional data with heterogeneity. It is now recognized that the $\ell_1$-penalty introduces non-negligible estimation bias, while a proper use of concave regularization may lead to estimators with refined convergence rates and oracle properties as the signal strengthens. Although folded concave penalized $M$-estimation with strongly convex loss functions have been well studied, the extant literature on quantile regression is relatively silent. The main difficulty is that the quantile loss is piecewise linear: it is non…

2021-09-12math.STstat.MEstat.TH

High-Dimensional Sparse Additive Hazards Regression

High-dimensional sparse modeling with censored survival data is of great practical importance, as exemplified by modern applications in high-throughput genomic data analysis and credit risk analysis. In this article, we propose a class of regularization methods for simultaneous variable selection and estimation in the additive hazards model, by combining the nonconcave penalized likelihood approach and the pseudoscore method. In a high-dimensional setting where the dimensionality can grow fast, polynomially or nonpolynomially, with the sample size, we establish the weak oracle property and ora…

2012-12-26math.STstat.MEstat.TH

多元计数回归中用于变量选择的支配超平面正则化

Dominating Hyperplane Regularization for Variable Selection in Multivariate Count Regression

针对高维多元计数回归中非凹似然函数难以优化的问题，提出支配超平面正则化方法，在MM框架下将SGL惩罚转化为加权岭惩罚，实现稳定优化与高效变量选择。

2025-04-07stat.APstat.ME

Hybrid Conditional Gradient - Smoothing Algorithms with Applications to Sparse and Low Rank Regularization

École Centrale Paris（巴黎中央理工学院）；KU Leuven（鲁汶大学）

We study a hybrid conditional gradient - smoothing algorithm (HCGS) for solving composite convex optimization problems which contain several terms over a bounded set. Examples of these include regularization problems with several norms as penalties and a norm constraint. HCGS extends conditional gradient methods to cases with multiple nonsmooth terms, in which standard conditional gradient methods may be difficult to apply. The HCGS algorithm borrows techniques from smoothing proximal methods and requires first-order computations (subgradients and proximity operations). Unlike proximal methods…

2014-04-15cs.LGmath.OCstat.ML

Solution Path Clustering with Adaptive Concave Penalty

Fast accumulation of large amounts of complex data has created a need for more sophisticated statistical methodologies to discover interesting patterns and better extract information from these data. The large scale of the data often results in challenging high-dimensional estimation problems where only a minority of the data shows specific grouping patterns. To address these emerging challenges, we develop a new clustering methodology that introduces the idea of a regularization path into unsupervised learning. A regularization path for a clustering problem is created by varying the degree of…

2014-04-24stat.MEstat.ML

Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization

Least squares fitting is in general not useful for high-dimensional linear models, in which the number of predictors is of the same or even larger order of magnitude than the number of samples. Theory developed in recent years has coined a paradigm according to which sparsity-promoting regularization is regarded as a necessity in such setting. Deviating from this paradigm, we show that non-negativity constraints on the regression coefficients may be similarly effective as explicit regularization if the design matrix has additional properties, which are met in several applications of non-negati…

2014-02-12math.STstat.MLstat.TH

A unified approach to model selection and sparse recovery using regularized least squares

Model selection and sparse recovery are two important problems for which many regularization methods have been proposed. We study the properties of regularization methods in both problems under the unified framework of regularized least squares with concave penalties. For model selection, we establish conditions under which a regularized least squares estimator enjoys a nonasymptotic property, called the weak oracle property, where the dimensionality can grow exponentially with sample size. For sparse recovery, we present a sufficient condition that ensures the recoverability of the sparsest s…

2009-09-03math.STstat.TH

Regularization Methods for High-Dimensional Instrumental Variables Regression With an Application to Genetical Genomics

In genetical genomics studies, it is important to jointly analyze gene expression data and genetic variants in exploring their associations with complex traits, where the dimensionality of gene expressions and genetic variants can both be much larger than the sample size. Motivated by such modern applications, we consider the problem of variable selection and estimation in high-dimensional sparse instrumental variables models. To overcome the difficulty of high dimensionality and unknown optimal instruments, we propose a two-stage regularization framework for identifying and estimating importa…

2014-03-18stat.APstat.ME

无脊正则化的高维弱工具变量回归模型的推断

Inference of high-dimensional weak instrumental variable regression models without ridge-regularization

针对高维弱工具变量回归，提出无需脊正则化和调节参数的二次型检验，结合最大型检验以应对密集与稀疏备择假设，对弱工具和异方差稳健，表现优于现有方法。

2025-04-29stat.ME