When restricted to a subspace, a nonsmooth function can be differentiable. It is known that for a nonsmooth convex function f and a point x, the Euclidean space can be decomposed into two subspaces: U, over which a special Lagrangian can be defined and has nice smooth properties and V, the orthogonal complement subspace of U. In this paper we generalize the definition of UV-decomposition and U-Lagrangian to the context of nonconvex functions, specifically that of a prox-regular function.
本文将Jascha \cite{sohl2014fast}和Lee \cite{lee2012proximal}最近的两个完全不同的工作分别推广和统一为一个,提出了优化两个凸函数(一个是大量光滑凸函数的平均值,另一个是非光滑凸函数)和的\textbf{prox} s \textbf{to}随机\textbf{N}牛顿型梯度(PROXTONE)方法.虽然最近提出的一组邻近随机梯度方法,包括MISO,Prox-SDCA,Prox-SVRG和SAG,以线性速率收敛,但PROXTONE包含二阶信息以获得更强的收敛结果,它不仅在目标函数的值中实现线性收敛速率,而且在\emph{solution}中也实现线性收敛速率。证明简单直观,所得结果和技巧可作为研究二阶信息近似随机方法的一个开端。
Constrained Bi-Level Optimization: Proximal Lagrangian Value function Approach and Hessian-free Algorithm
National Center for Applied Mathematics Shenzhen(深圳国家应用数学中心);Southern University of Science and Technology(南方科技大学);CETC Key Laboratory of Smart City Modeling Simulation and Intelligent Technology(中国电子科技集团智慧城市建模仿真与智能技术重点实验室);The Smart City Research Institute of CETC(中国电子科技集团智慧城市研究院);University of Victoria(维多利亚大学)
Prox-regularity is a generalization of convexity that includes all C2, lower-C2, strongly amenable and primal-lower-nice functions. The study of prox- regular functions provides insight on a broad spectrum of important functions. Parametrically prox-regular (para-prox-regular) functions are a further extension of this family, produced by adding a parameter. Such functions have been shown to play a key role in understanding stability of minimizers in optimization problems . This document discusses para-prox- regular functions in Rn: we begin with some basic examples of para-prox- regular functi…
Developing Lagrangian-based Methods for Nonsmooth Nonconvex Optimization
Institute of Operational Research and Analytics, National University of Singapore(新加坡国立大学运营研究与分析研究所);School of Computer and Computing Science, Hangzhou City University(杭州城市大学计算机与计算科学学院)
Stochastic Perturbation of Sweeping Processes Driven by Continuous Uniformly Prox-Regular Moving Sets
Universidad de Chile(智利大学);Université de Lorraine(洛林大学);Universidad de O’Higgins(奥希金斯大学);Centro de Modelamiento Matemático (CNRS UMI 2807)(数学建模中心(法国国家科学研究中心联合国际单位2807))