arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.02155math.OC
收藏

Hölderian误差界下简单双层优化的惩罚类方法

Penalty-based Methods for Simple Bilevel Optimization under Hölderian Error Bounds

Pengyu Chen, Xu Shi, Rujun Jiang, Jiulin Wang

备注：
Accepted by NeurIPS 2024
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有简单双层优化方法收敛性差、速率慢或假设强的问题，提出适配不同光滑凸性假设的惩罚框架，在Hölderian误差界下得到多种场景的复杂度结果，实验验证了算法有效性。

摘要：

本文研究简单双层优化问题，即在凸下层目标的最优解集上最小化上层目标。现有求解该类问题的方法要么仅能保证渐近收敛，要么收敛速度为慢的次线性速率，抑或需要较强的假设条件。为应对这些挑战，我们提出一种惩罚框架，明确了原问题的近似解与其重构形式的近似解之间的关联。该框架可适配关于光滑性与凸性的不同假设，支持应用具有不同复杂度结果的特定方法。具体而言，当上下层目标均为复合凸函数时，在α-Hölderian误差界条件及若干温和假设下，我们的算法可在O(√(1/ε^{max{α,β}}))次迭代内，对任意β>0得到原问题的(ε,ε^β)-最优解。若上层目标的光滑部分为强凸函数，该结果可进一步改进。我们还建立了上下层目标为一般非光滑函数时的复杂度结果。数值实验验证了所提算法的有效性。

英文摘要：

This paper investigates simple bilevel optimization problems where we minimize an upper-level objective over the optimal solution set of a convex lower-level objective. Existing methods for such problems either only guarantee asymptotic convergence, have slow sublinear rates, or require strong assumptions. To address these challenges, we propose a penalization framework that delineates the relationship between approximate solutions of the original problem and its reformulated counterparts. This framework accommodates varying assumptions regarding smoothness and convexity, enabling the application of specific methods with different complexity results. Specifically, when both upper- and lower-level objectives are composite convex functions, under an $α$-H{ö}lderian error bound condition and certain mild assumptions, our algorithm attains an $(ε,ε^β)$-optimal solution of the original problem for any $β> 0$ within $\mathcal{O}\left(\sqrt{{1}/{ε^{\max\{α,β\}}}}\right)$ iterations. The result can be improved further if the smooth part of the upper-level objective is strongly convex. We also establish complexity results when the upper- and lower-level objectives are general nonsmooth functions. Numerical experiments demonstrate the effectiveness of our algorithms.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑