基于外推的非光滑随机零阶优化直接搜索方法
Extrapolation-based Direct Search for Nonsmooth Stochastic Zeroth-Order Optimization
AI总结：
本文提出一种适用于非光滑随机零阶优化的基于外推的直接搜索方法，证明其收敛性与复杂度界，经DFO基准实验验证性能具竞争力。
AI中文摘要：
我们提出并分析了一种用于无约束零阶最小化的随机直接搜索方法，适用于局部利普希茨、可能非光滑的目标函数。该方法将随机 polling 方向与基于 p 阶充分下降检验的随机外推线搜索相结合。在随机估计的条件精度假设下，我们证明了其几乎必然收敛到 Clarke 平稳点。我们进一步建立了期望迭代复杂度界，具体而言，利用上鞅停止时间论证，证明了在期望意义下，使用 O(max{r⁻ᵖ, ε⁻ᵖ/⁽ᵖ⁻¹⁾}) 次迭代足以达到 (r, ε)-Goldstein 平稳点。此外，我们推导了相应的期望测试点复杂度界，阶数为 O(ε¹⁻ⁿ max{r⁻ᵖ, ε⁻ᵖ/⁽ᵖ⁻¹⁾})。据我们所知，这是针对非光滑随机情形下基于外推的直接搜索方法的首次收敛性与期望复杂度分析。在 DFO 基准套件上的数值实验表明，该方法相较于成熟的随机直接搜索方法具有竞争力的性能。
英文摘要：
We propose and analyze a stochastic direct-search method for unconstrained zeroth-order minimization of locally Lipschitz, possibly nonsmooth, objectives. The method combines random polling directions with a stochastic extrapolating line search based on a sufficient-decrease test of order $p$. Under conditional accuracy assumptions on the stochastic estimates, we prove almost-sure convergence to Clarke stationary points. We further establish an expected iteration complexity bound. Specifically, using a supermartingale stopping-time argument, we prove that $\mathcal O\left( \max\left\{ r^{-p}, \varepsilon^{-p/(p-1)} \right\} \right) $ iterations are sufficient in expectation to reach an $(r,\varepsilon)$-Goldstein stationary point. Moreover, we derive a corresponding expected tested-point complexity bound of order $\mathcal O\bigl(\varepsilon^{1-n} \max\{r^{-p},\varepsilon^{-p/(p-1)}\}\bigr)$. To the best of our knowledge, this is the first convergence and expected-complexity analysis for an extrapolation-based direct-search method in a nonsmooth stochastic setting. Numerical experiments on a DFO benchmark suite highlight competitive performance against well-established stochastic direct-search methods.
