AI 中文总结
研究随机非凸简单双层优化问题，提出SDBPG方法，通过自适应扰动对偶公式解决对偶乘子无界问题，在特定假设下能高效找到驻点，还开发了改进变体，给出了该问题首个明确的驻点保证。
AI 中文摘要
我们研究仅通过随机梯度预言机访问的具有光滑、可能非凸的上下层目标的随机简单双层优化。关键挑战是下层约束引起的对偶乘子在下层驻点附近可能变得无界。为解决此问题，我们提出随机动态屏障扰动梯度（SDBPG）方法，它通过自适应扰动对偶公式来正则化这种退化。在温和的罕见访问假设下，SDBPG能在\(\mathcal{O}(\max\{\epsilon_f^{-2},\epsilon_g^{-2}\})\)次迭代中找到\((\epsilon_f,\epsilon_g)\) - 驻点，上下层目标的样本梯度复杂度分别为\(\mathcal{O}(\epsilon^{-4})\)和\(\mathcal{O}(\epsilon^{-6})\)。我们还开发了PR - SDBPG和VR - PR - SDBPG。据我们所知，这些是随机非凸 - 非凸简单双层优化的首个明确的\((\epsilon_f,\epsilon_g)\) - 驻点保证。
英文摘要
We study stochastic simple bilevel optimization with smooth, possibly nonconvex upper- and lower-level objectives accessed only through stochastic gradient oracles. A key challenge is that the dual multiplier induced by the lower-level constraint may become unbounded near lower-level stationary points, invalidating bounded-dual analyses and destabilizing stochastic gradient estimates. To address this, we propose \emph{Stochastic Dynamic Barrier Perturbed Gradient} (SDBPG), a single-loop method that adaptively perturbs the dual formulation to regularize this degeneracy. The perturbation stabilizes the multiplier and yields controlled bias and variance even near the lower-level stationarity region. Under a rare-visit assumption governed by a parameter $δ\in (0, \tfrac{1}{2}]$, SDBPG finds an $(ε, ε)$-stationary point in $\mathcal{O}(ε^{-1/δ})$ iterations, with sample gradient complexities $\mathcal{O}(ε^{-2/δ})$ and $\mathcal{O}(ε^{-3/δ})$ for the upper- and lower-level objectives, where larger $δ$ corresponds to rarer visits to the bad region describing the negative alignment between the two objectives when the lower-level gradient is small. We further develop PR-SDBPG, a penalty-regularized variant that eliminates the rare-visit assumption, and VR-PR-SDBPG, which improves the resulting sample complexities entirely through variance reduction. To our knowledge, these are the first explicit $(ε_f,ε_g)$-stationarity guarantees for stochastic nonconvex-nonconvex simple bilevel optimization.
CommentsThe new version resolves the issue in the proof of Theorem 3.3 and modifies Assumption 3.1 to impose the rare visit assumption on the true bad region only
