高维正则化M-估计的精确误差分析
Precise Error Analysis of Regularized M-estimators in High-dimensions
摘要:
用于从有噪声的线性测量估计未知信号的流行方法是通过求解所谓的\emph{regularized M-estimator},其最小化凸损失函数和凸(典型地,非平滑)正则化器的加权组合。我们准确地预测在高维比例制度,这样的估计的平方误差性能。假设随机测量矩阵具有Iid高斯项,仅对损失函数、正则化器以及噪声和信号分布施加最小且相当温和的正则性条件。我们发现,误差收敛概率的非平凡的限制,这是一个极大极小凸凹优化问题的解决方案,四个标量优化变量。我们确定了一个新的汇总参数,称为预期莫罗包络发挥核心作用的错误表征。结果的\emph{precise}性质允许在正则化M估计器的不同实例之间进行准确的性能比较,并允许最佳地调整所涉及的参数(例如正则化器参数、测量次数)。我们证明的关键成分是\emph{Convex Gaussian Min-max Theorem}(CGMT),这是一个严格的和加强版本的经典高斯比较不等式,证明了戈登在1988年。
英文摘要:
A popular approach for estimating an unknown signal from noisy, linear measurements is via solving a so called \emph{regularized M-estimator}, which minimizes a weighted combination of a convex loss function and of a convex (typically, non-smooth) regularizer. We accurately predict the squared error performance of such estimators in the high-dimensional proportional regime. The random measurement matrix is assumed to have entries iid Gaussian, only minimal and rather mild regularity conditions are imposed on the loss function, the regularizer, and on the noise and signal distributions. We show that the error converges in probability to a nontrivial limit that is given as the solution to a minimax convex-concave optimization problem on four scalar optimization variables. We identify a new summary parameter, termed the Expected Moreau envelope to play a central role in the error characterization. The \emph{precise} nature of the results permits an accurate performance comparison between different instances of regularized M-estimators and allows to optimally tune the involved parameters (e.g. regularizer parameter, number of measurements). The key ingredient of our proof is the \emph{Convex Gaussian Min-max Theorem} (CGMT) which is a tight and strengthened version of a classical Gaussian comparison inequality that was proved by Gordon in 1988.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。