arXiv 2304.047782023-04math.OCcs.LG

带函数约束的随机变分不等式问题的一阶方法

First-order methods for Stochastic Variational Inequality problems with Function Constraints

Southern Methodist University（南卫理公会大学）；Shanghai University of Finance and Economics（上海财经大学）

Digvijay Boob, Qi Deng, Mohammad Khalafi

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文针对带函数约束的单调变分不等式问题，提出了一阶方法AdOpEx、P-OpEx和OpConEx，分别达到最优或改进的收敛速率，并扩展到鞍点问题。

摘要：

单调变分不等式（VI）是一个通用模型，在工程和科学领域有重要应用。在许多情况下，VI问题伴随有函数约束，这些约束可能是数据驱动的，使得通常的投影算子难以计算。本文提出了新颖的一阶方法，用于在光滑或非光滑设置下，且算子和约束可能是随机的情况下，解决函数约束变分不等式（FCVI）问题。我们引入了AdOpEx方法，该方法在光滑确定性设置中对FCVI的KKT算子进行算子外推。由于该算子在拉格朗日乘子方面不是一致Lipschitz连续的，我们采用自适应双时间尺度算法，使得乘子有界，并达到最优的$O(1/T)$收敛速率。对于非光滑和随机VI，我们对AdOpEx方法进行了设计修改，并提出了一种新颖的P-OpEx方法，该方法采用部分外推。当算子和约束都是随机或非光滑时，它以$O(1/\sqrt{T})$的速率收敛。该方法在噪声和函数约束的Lipschitz常数方面具有次优依赖性。我们提出了一种约束外推方法，即OpConEx方法，该方法将这种依赖性提高了一个数量级。我们所有的算法都容易扩展到具有耦合原始变量和对偶变量的函数约束的鞍点问题，同时保持相同的复杂度结果。据我们所知，我们所有的复杂度结果在文献中都是新的。

英文摘要：

The monotone Variational Inequality (VI) is a general model with important applications in various engineering and scientific domains. In numerous instances, the VI problems are accompanied by function constraints that can be data-driven, making the usual projection operator challenging to compute. This paper presents novel first-order methods for the function-constrained Variational Inequality (FCVI) problem in smooth or nonsmooth settings with possibly stochastic operators and constraints. We introduce the AdOpEx method, which employs an operator extrapolation on the KKT operator of the FCVI in a smooth deterministic setting. Since this operator is not uniformly Lipschitz continuous in the Lagrange multipliers, we employ an adaptive two-timescale algorithm leading to bounded multipliers and achieving the optimal $O(1/T)$ convergence rate. For the nonsmooth and stochastic VIs, we introduce design changes to the AdOpEx method and propose a novel P-OpEx method that takes partial extrapolation. It converges at the rate of $O(1/\sqrt{T})$ when both the operator and constraints are stochastic or nonsmooth. This method has suboptimal dependence on the noise and Lipschitz constants of function constraints. We propose a constraint extrapolation approach leading to the OpConEx method that improves this dependence by an order of magnitude. All our algorithms easily extend to saddle point problems with function constraints that couple the primal and dual variables while maintaining the same complexity results. To the best of our knowledge, all our complexity results are new in the literature

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文