arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.04552cs.LGstat.ML

乐观在线镜像下降：桥接随机与对抗在线凸优化

Optimistic Online Mirror Descent for Bridging Stochastic and Adversarial Online Convex Optimization

National Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；The University of Tokyo（东京大学）；Paradigm Inc.（第四范式公司）；School of Artificial Intelligence, Nanjing University（南京大学人工智能学院）

Sijia Chen, Yu-Jie Zhang, Wei-Wei Tu, Peng Zhao, Lijun Zhang

备注：
v3 substantially improves the presentation and has a few improvements, including the regret bound for strongly convex functions; v2 is an extended version that enriches the content with improved regret bounds for strongly convex functions, discussions on the optimism design for dynamic regret minimization, and extensions to non-smooth scenarios; v1 is the ICML 2023 conference version
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

该论文研究SEA模型下乐观在线镜像下降（OMD）的遗憾界，在凸、强凸和指数凹光滑函数下分别获得与FTRL相当或更优的静态遗憾界，并首次给出动态遗憾保证，同时提出处理非光滑函数的隐式更新算法。

摘要：

Sachs等人[2022]引入了随机扩展对抗（SEA）模型，作为随机与对抗在线凸优化之间的插值。在光滑性条件下，他们证明了乐观跟随正则化领导者（FTRL）的期望遗憾依赖于凸函数的累积随机方差$σ_{1:T}^2$和累积对抗变化$Σ_{1:T}^2$。他们还为强凸函数提供了基于最大随机方差$σ_{\max}^2$和最大对抗变化$Σ_{\max}^2$的稍弱界。受其工作启发，我们研究了乐观在线镜像下降（OMD）在SEA模型下的理论保证。对于凸且光滑的函数，我们获得了相同的$\mathcal{O}(\sqrt{σ_{1:T}^2}+\sqrt{Σ_{1:T}^2})$遗憾界，且不要求单个函数的凸性。对于强凸且光滑的函数，我们建立了$\mathcal{O}((σ_{\max}^2 + Σ_{\max}^2) \log (σ_{1:T}^2+Σ_{1:T}^2))$的界，优于他们的$\mathcal{O}((σ_{\max}^2 + Σ_{\max}^2) \log T)$结果。对于指数凹且光滑的函数，我们实现了新的$\mathcal{O}(d\log(σ_{1:T}^2+Σ_{1:T}^2))$界。得益于OMD框架，我们将工作扩展到动态遗憾最小化以及在线函数非光滑的场景。我们为具有凸且光滑函数的SEA模型建立了首个动态遗憾保证，在非平稳场景中该保证比静态遗憾界更优。此外，为了处理SEA模型中的非光滑凸函数，我们提出了基于具有隐式更新的乐观OMD的新算法，该算法在无光滑性条件下可证明地获得静态遗憾和动态遗憾保证。

英文摘要：

Stochastically Extended Adversarial (SEA) model is introduced by Sachs et al. [2022] as an interpolation between stochastic and adversarial online convex optimization. Under the smoothness condition, they demonstrate that the expected regret of optimistic follow-the-regularized-leader (FTRL) depends on the cumulative stochastic variance $σ_{1:T}^2$ and the cumulative adversarial variation $Σ_{1:T}^2$ for convex functions. They also provide a slightly weaker bound based on the maximal stochastic variance $σ_{\max}^2$ and the maximal adversarial variation $Σ_{\max}^2$ for strongly convex functions. Inspired by their work, we investigate the theoretical guarantees of optimistic online mirror descent (OMD) for the SEA model. For convex and smooth functions, we obtain the same $\mathcal{O}(\sqrt{σ_{1:T}^2}+\sqrt{Σ_{1:T}^2})$ regret bound, without the convexity requirement of individual functions. For strongly convex and smooth functions, we establish an $\mathcal{O}((σ_{\max}^2 + Σ_{\max}^2) \log (σ_{1:T}^2+Σ_{1:T}^2))$ bound, better than their $\mathcal{O}((σ_{\max}^2 + Σ_{\max}^2) \log T)$ result. For exp-concave and smooth functions, we achieve a new $\mathcal{O}(d\log(σ_{1:T}^2+Σ_{1:T}^2))$ bound. Owing to the OMD framework, we broaden our work to study dynamic regret minimization and scenarios where the online functions are non-smooth. We establish the first dynamic regret guarantee for the SEA model with convex and smooth functions, which is more favorable than static regret bounds in non-stationary scenarios. Furthermore, to deal with non-smooth and convex functions in the SEA model, we propose novel algorithms building on optimistic OMD with an implicit update, which provably attain static regret and dynamic regret guarantees without smoothness conditions.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑