发表机构
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出单循环一阶算法MRT-FD，用于非凸-强凸随机双层优化，通过有限差分近似二阶导数，达到最优复杂度O(ε^{-4-2/p})并证明匹配下界。
AI 中文摘要
我们研究了在随机一阶预言机下的非凸-强凸双层优化问题。我们引入了MRT-FD，一种单循环一阶方法，它同时跟踪上层变量、下层解以及由超目标隐式微分产生的辅助响应。MRT-FD每次迭代对每个变量进行一次更新，并使用p阶有限差分来近似二阶导数作用。对于下层变量中任意固定的有限光滑阶数p≥1，MRT-FD使用O(ε^{-4-2/p})次随机梯度查询即可找到ε-平稳点。我们还证明了匹配的Ω(ε^{-4-2/p})预言机下界。该下界构造从一个具有更强随机预言机的困难非凸最小化链开始，并通过正弦耦合与一个标量下层变量将其提升为双层问题。因此，对于每个固定的有限p，对ε的依赖是最优的，从而缩小了该随机一阶预言机设置中的上-下复杂度差距。
英文摘要
We study nonconvex--strongly-convex bilevel optimization under a stochastic first-order oracle. We introduce MRT-FD, a single-loop first-order method that simultaneously tracks the upper-level variable, the lower-level solution, and the auxiliary response arising from implicit differentiation of the hyperobjective. MRT-FD performs one update of each variable per iteration and approximates the second-order derivative actions using order-$p$ finite differences. For any fixed finite smoothness order $p\ge1$ in the lower-level variable, MRT-FD finds an $\varepsilon$-stationary point using $\mathcal{O}(\varepsilon^{-4-2/p})$ stochastic gradient queries. We also prove a matching $Ω(\varepsilon^{-4-2/p})$ oracle lower bound. The lower-bound construction starts from a hard nonconvex minimization chain with a stronger stochastic oracle, and lifts it to a bilevel problem through a sinusoidal coupling with a scalar lower-level variable. Consequently, the dependence on $\varepsilon$ is optimal for every fixed finite $p$, closing the upper--lower complexity gap in this stochastic first-order oracle setting.
