用于稀疏学习的I-LAMM：同时控制算法复杂度和统计误差
I-LAMM for Sparse Learning: Simultaneous Control of Algorithmic Complexity and Statistical Error
摘要：
我们提出了一个计算框架，称为迭代局部自适应优化最小化（I-LAMM），同时控制算法的复杂性和统计误差时，拟合高维模型。I-LAMM是一个两阶段的算法实现的局部线性逼近的家庭折叠凹惩罚准似然。第一阶段解决一个凸规划与粗精度公差，以获得一个粗略的初始估计，这是进一步细化在第二阶段迭代求解一系列的凸规划具有较小的精度公差。理论上，我们建立了一个相变：第一阶段具有次线性迭代复杂度，而第二阶段实现了改进的线性收敛速率。虽然这个框架是完全算法，它提供了最佳的统计性能和控制算法复杂性的一个大家庭的非凸最优化问题的解决方案。通过一个收缩性质，清楚地证明了迭代对统计误差的影响。我们的理论依赖于一个本地化版本的稀疏/限制特征值条件，这使我们能够分析一个大家庭的损失和罚函数，并提供最优性保证非常弱的假设下（例如，I-LAMM需要弱得多的最小信号强度比其他程序）。彻底的数值结果提供支持所得到的理论。
英文摘要：
We propose a computational framework named iterative local adaptive majorize-minimization (I-LAMM) to simultaneously control algorithmic complexity and statistical error when fitting high dimensional models. I-LAMM is a two-stage algorithmic implementation of the local linear approximation to a family of folded concave penalized quasi-likelihood. The first stage solves a convex program with a crude precision tolerance to obtain a coarse initial estimator, which is further refined in the second stage by iteratively solving a sequence of convex programs with smaller precision tolerances. Theoretically, we establish a phase transition: the first stage has a sublinear iteration complexity, while the second stage achieves an improved linear rate of convergence. Though this framework is completely algorithmic, it provides solutions with optimal statistical performances and controlled algorithmic complexity for a large family of nonconvex optimization problems. The iteration effects on statistical errors are clearly demonstrated via a contraction property. Our theory relies on a localized version of the sparse/restricted eigenvalue condition, which allows us to analyze a large family of loss and penalty functions and provide optimality guarantees under very weak assumptions (For example, I-LAMM requires much weaker minimal signal strength than other procedures). Thorough numerical results are provided to support the obtained theory.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
