用于一般凸-凹鞍点问题的、带有两个凸组合和线搜索的新原始-对偶算法

中文摘要 AI 辅助

凸-凹鞍点问题广泛存在于机器学习、图像处理、经济学及均衡问题等多个领域，原始-对偶算法是求解这类问题的高效且强大的框架。凸组合已成为原始-对偶算法的关键加速技术，该技术的整合近期使这类算法成为热门研究方向。凸组合参数的选择对对应算法的理论分析和数值性能均有显著影响，但理论对该参数的要求有时与数值实验的建议不一致，例如理论分析常要求参数较小，而数值实验有时倾向于更大的参数值。为解决该不一致问题并进一步推进带凸组合的原始-对偶算法，本文提出一种基于两个凸组合的新策略，将其整合到原始-对偶框架中，得到用于一般凸-凹鞍点问题的新原始-对偶算法NPDAL-n。NPDAL-n中提出的两个凸组合确保凸组合参数的允许范围主要由理论考虑决定，几乎无需考虑数值性能。通过严格的李雅普诺夫能量下降分析，在标准假设下，本文证明NPDAL-n具有全局收敛性和次线性遍历收敛率$\tilde{\text{O}}(1/N)$；当初始函数为强凸时，本文开发了NPDAL-n的加速版本，其达到最优$\tilde{\text{O}}(1/N^2)$收敛率。

英文摘要

Convex-concave saddle-point problems are ubiquitous across diverse domains, including machine learning, image processing, economics, and equilibrium problems. Primal-dual algorithms provide a highly effective and powerful framework for convex-concave saddle-point problems. Convex combination has become a crucial acceleration technique for primal-dual algorithms, and the integration of this technique has recently made these algorithms a highly active research topic. The choice of the convex combination parameter often has a significant impact on both the theoretical analysis and the numerical performance of the corresponding algorithms. However, the requirements on this parameter imposed by theory are sometimes inconsistent with those suggested by numerical experiments. For instance, theoretical analysis often requires the parameter to be small, while numerical experiments tend sometimes to favor larger values. To address this inconsistency and further advance primal-dual algorithms with convex combination, we develop a novel strategy based on two convex combinations, integrate it into a primal-dual framework, and propose a new primal-dual algorithm with linesearch, termed NPDAL-n, for general convex-concave saddle-point problems. The proposed two convex combinations in NPDAL-n ensure that the permissible range of the convex combination parameters is mainly determined by theoretical considerations, with little regard for numerical performance. Through rigorous Lyapunov energy descent analysis, we establish the global convergence and a sublinear ergodic convergence rate of $\mathcal{O}(1/N)$ for NPDAL-n under standard assumptions. When the primal function is strongly convex, we develop an accelerated version of NPDAL-n that achieves an optimal $\mathcal{O}(1/N^2)$ rate.

发表机构

• School of Mathematics and Statistics, Guizhou University（贵州大学数学与统计学院）

机构由 AI 辅助整理，请以论文原文为准。