As a starting point of our research, we show that, for a fixed order $γ\geq 1$, each local minimizer of a rather general nonsmooth optimization problem in Euclidean spaces is either M-stationary in the classical sense (corresponding to stationarity of order $1$), satisfies stationarity conditions in terms of a coderivative construction of order $γ$, or is asymptotically stationary with respect to a critical direction as well as order $γ$ in a certain sense. By ruling out the latter case with a constraint qualification not stronger than directional metric subregularity, we end up with new necessary optimality conditions comprising a mixture of limiting variational tools of orders $1$ and $γ$. These abstract findings are carved out for the broad class of geometric constraints and $γ:=2$, and visualized by examples from complementarity-constrained and nonlinear semidefinite optimization. As a byproduct of the particular setting $γ:=1$, our general approach yields new so-called directional asymptotic regularity conditions which serve as constraint qualifications guaranteeing M-stationarity of local minimizers. We compare these new regularity conditions with standard constraint qualifications from nonsmooth optimization. Further, we extend directional concepts of pseudo- and quasi-normality to arbitrary set-valued mappings. It is shown that these properties provide sufficient conditions for the validity of directional asymptotic regularity. Finally, a novel coderivative-like variational tool is used to construct sufficient conditions for the presence of directional asymptotic regularity. For geometric constraints, it is illustrated that all appearing objects can be calculated in terms of initial problem data.
Directional first order approach for a class of bilevel programs
School of Mathematics, Hunan University(湖南大学数学学院);School of Artificial Intelligence, Southern University of Science and Technology(南方科技大学人工智能学院);Department of Mathematics and Statistics, University of Victoria(维多利亚大学数学与统计系);Department of Mathematics, Southern University of Science and Technology(南方科技大学数学系)
AdaDGS: An adaptive black-box optimization method with a nonlocal directional Gaussian smoothing gradient
Oak Ridge National Laboratory(橡树岭国家实验室)
局部梯度指向无穷小邻域中最陡斜率的方向。当损失是多模态的时,由局部梯度引导的优化器经常陷入局部最优。最近在(Zhang et al.,2020),并用于定义一个真正的非局部梯度,称为DGS梯度,用于高维黑盒优化。有希望的结果表明,取代传统的局部梯度与DGS梯度可以显着提高基于梯度的方法在优化高度多模态损失函数的性能。然而,DGS梯度的最佳性能可能依赖于两个重要超参数的微调,即,平滑半径和学习率。在本文中,我们提出了一个简单的,但巧妙的和有效的自适应方法与DGS梯度优化,消除了超参数微调的需要。由于DGS梯度通常指向良好的搜索方向,因此我们沿着DGS方向执行线搜索以确定每次迭代的步长。学习的步长反过来会告诉我们周围区域中函数景观的规模,基于此,我们相应地调整平滑半径以进行下一次迭代。我们提出了高维基准函数,翼型设计问题和游戏内容生成问题的实验结果。AdaDGS方法已经显示出优于几种最先进的黑盒优化方法的性能。
Sufficient Conditions for Metric Subregularity of Constraint Systems with Applications to Disjunctive and Ortho-Disjunctive Programs
本文研究一般最优化问题的度量次正则约束规格(MSCQ),重点是非凸设置。我们详细阐述了方向伪正态性和准正态性的概念,最近由Bai等人(SIAM J. Opt.,2019年),它结合了标准的方法,通过伪和准常态与现代工具的方向变分分析。我们专注于应用程序的析取程序,其中(方向)伪正规性的特点是通过一个极值条件。这反过来又产生了有效的工具来验证伪正规性和MSCQ,其中包括但不限于Robinson关于多面体多函数的结果和Gfrerer关于度量次正则性的二阶充分条件。最后,我们完善我们的研究,通过定义一类新的ortho-disjunctive程序,其中包括突出的最优化问题,如数学规划的互补性,消失或切换约束。