非凸广义线性模型在高维中的渐进行为：复制公式的一个证明
Asymptotics of Non-Convex Generalized Linear Models in High-Dimensions: A proof of the replica formula
AI总结：
本文通过严格证明非凸GLMs的复制对称公式，展示了其在高维优化中的有效性，并通过Tukey损失、负则正则化和线性化AMP算法的应用，验证了统计物理预测在非凸问题中的正确性。
摘要：
高维优化对于广义线性模型（GLMs）在高斯数据下的分析特性已成为统计学和概率论近年来的一个核心关注点。尽管凸情况，如LASSO、岭回归和逻辑回归，已通过多种技术进行了广泛研究，而非凸情况仍远未被充分理解，尽管其重要性不言而喻。一个非严谨的统计物理框架已为高维优化问题的行为提供了显著的预测，但将其严格建立为非凸问题的有效性仍是一个根本性挑战。在本工作中，我们通过开发一个系统框架，严格证明非凸GLMs的复制对称公式，并精确确定这些公式有效的条件。令人瞩目的是，严格推导出的复制对称预测与物理学家的猜想以及所谓的复制子条件完全一致。我们方法的原创性在于连接了两种强大的理论工具：用于提供精确下限的高斯最小-最大定理，以及被证明在算法上达到这些下限的近似消息传递（AMP）。我们通过重要应用展示了该框架的实用性：（i）通过证明在ε污染数据模型下Tukey损失优于更常用的Huber损失的最优性；（ii）在高维非凸回归中建立负则正则化的最优性；（iii）表征线性化AMP算法的性能极限。通过严格验证非凸情况下统计物理预测的有效性，我们旨在开辟新的途径，以分析超出凸范围的日益复杂的优化景观。
英文摘要：
The analytic characterization of the high-dimensional behavior of optimization for Generalized Linear Models (GLMs) with Gaussian data has been a central focus in statistics and probability in recent years. While convex cases, such as the LASSO, ridge regression, and logistic regression, have been extensively studied using a variety of techniques, the non-convex case remains far less understood despite its significance. A non-rigorous statistical physics framework has provided remarkable predictions for the behavior of high-dimensional optimization problems, but rigorously establishing their validity for non-convex problems has remained a fundamental challenge. In this work, we address this challenge by developing a systematic framework that rigorously proves replica-symmetric formulas for non-convex GLMs and precisely determines the conditions under which these formulas are valid. Remarkably, the rigorous replica-symmetric predictions align exactly with the conjectures made by physicists, and the so-called replicon condition. The originality of our approach lies in connecting two powerful theoretical tools: the Gaussian Min-Max Theorem, which we use to provide precise lower bounds, and Approximate Message Passing (AMP), which is shown to achieve these bounds algorithmically. We demonstrate the utility of this framework through significant applications: (i) by proving the optimality of the Tukey loss over the more commonly used Huber loss under a $\varepsilon$ contaminated data model, (ii) establishing the optimality of negative regularization in high-dimensional non-convex regression and (iii) characterizing the performance limits of linearized AMP algorithms. By rigorously validating statistical physics predictions in non-convex settings, we aim to open new pathways for analyzing increasingly complex optimization landscapes beyond the convex regime.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
