悲观双层优化问题的单循环梯度算法

中文摘要 AI 辅助

双层优化近年来受到越来越多的关注，特别是在高效数值方法的发展方面。尽管在乐观双层优化方面取得了实质性进展，但悲观双层优化（PBO）的探索仍然较少，尤其是完全一阶、单循环基于梯度的方法的设计。为了解决这一空白，我们通过重构、惩罚和正则化提出了PBO的光滑近似，并建立了关于最小点和稳定性的收敛保证。在此框架基础上，我们分别针对确定性和随机PBO开发了两种单循环算法。两者仅使用一阶梯度信息，避免二阶导数和内循环子问题求解。我们为所提出的算法建立了非渐近收敛速率，以提供理论保证。通过对合成和实际问题实例的系统性实证研究，我们证明了我们的算法高度有效和高效。特别是，我们在垃圾邮件分类和智能预测-然后-优化上的结果进一步展示了PBO相对于其经典乐观双层对应物的优势，突显了其在实际建模和提供稳健解决方案方面的强大潜力。

英文摘要

Bilevel optimization has recently attracted growing attention, particularly in the development of efficient numerical methods. Despite substantial progress on optimistic bilevel optimization, pessimistic bilevel optimization (PBO) remains much less explored, especially the design of fully first-order, single-loop gradient-based methods. To address this gap, we propose a smooth approximation of PBO through reformulation, penalization and regularization, and establish convergence guarantees in terms of both minimizers and stationarity. Building on this framework, we then develop two single-loop algorithms for deterministic and stochastic PBOs, respectively. Both use only first-order gradient information and avoid second-order derivatives and inner-loop subproblem solves. Non-asymptotic convergence rates for the proposed algorithms are established to provide theoretical guarantees. Through a systematic empirical study of both synthetic and practical problem instances, we demonstrate that our algorithms are highly effective and efficient. In particular, our results on spam classification and Smart Predict-then-Optimize further illustrate PBO's advantages over its classical optimistic bilevel counterpart, highlighting its strong potential for practical modeling and the delivery of robust solutions.

发表机构

• Southern University of Science and Technology（南方科技大学）
• University of Pittsburgh（匹兹堡大学）
• National Center for Applied Mathematics Shenzhen（深圳应用数学国家研究中心）

机构由 AI 辅助整理，请以论文原文为准。