非凸非凹极小极大问题的一种序贯平滑主化随机逼近方法
A sequential smoothing majorant stochastic approximation method for nonconvex nonconcave minimax problems
中文总结 AI 辅助
提出序贯平滑主化随机逼近（SMSA）方法求解非凸非凹极小极大问题，通过新主化逼近实现$O(\beta_N^2)$精度，并证明稳定点一致性与数值优势。
中文摘要 AI 辅助
我们针对非凸-非凹极小极大优化问题提出了一种序贯平滑主化随机逼近（SMSA）方法。为克服内层极大化问题的非凹性，我们引入了一种新的主化随机逼近方法，其精度界为$O\left(\beta_N^2\right)$，其中$\beta_N$为样本覆盖半径。该精度界显著优于标准随机逼近的$O\left(\beta_N\right)$逼近界。此外，我们建立了全局最优值和极小化子集的非渐近界，并证明了当$\beta_N \downarrow 0$几乎必然成立时，Clarke稳定点的一致性。我们证明了SMSA方法生成的序列有界，且返回的点是主化随机逼近模型的近似Clarke稳定点。在合成玩具示例和两个UCI数据集上的鲁棒逻辑回归数值实验表明，相对于标准采样逼近，该方法具有更高的逼近保真度和更低的平均鲁棒测试损失。
英文摘要
We propose a sequential smoothing majorant stochastic approximation (SMSA) method for nonconvex-nonconcave minimax optimization problems. To overcome the nonconcavity of the inner maximization problem, we introduce a new majorant stochastic approximation with an $O\left(β_N^2\right)$ accuracy bound, where $β_N$ is the sample coverage radius. The accuracy bound significantly improves upon the $O\left(β_N\right)$ approximation bound for the standard stochastic approximation. Moreover, we establish nonasymptotic bounds for both global optimal values and minimizer sets, and prove consistency for the Clarke stationary points, as $β_N \downarrow 0$ almost surely. We show that the generated sequence by the SMSA method is bounded, and that the returned point is an approximate Clarke stationary point of the majorant stochastic approximation models. Numerical experiments on a synthetic toy example and robust logistic regression on two UCI datasets demonstrate improved approximation fidelity and lower mean robust test losses relative to the standard sampled approximation.
