本文从拓扑的角度研究了稀疏约束非线性最优化问题。特别关注Burdakov et al.(2016)的M-平稳点。引入了非退化M-平稳点并定义了它们的M-指标。我们证明了所有的M-平稳点是一般非退化的。特别地,稀疏约束在通用SCNO的所有局部最小值处是活动的。详细讨论了它与其它平稳性概念的关系,如S-平稳性、基本可行性、CW-极小性等。通过这样做,由于不同的平稳性概念的点的不稳定性和退化的问题突出。M-平稳性的概念允许沿着莫尔斯理论的路线充分描述SCNO的全局结构。为此,我们研究了低水平集通过M-平稳点时的拓扑变化。作为SCNO的新颖之处,需要附加尺寸等于M指数的多个单元。这个有趣的事实与之前考虑的其他最优化问题形成强烈对比,在其他问题中,只有一个单元就足够了。因此,我们得到了一个Morse关系SCNO,它涉及的局部极小值和M-指数等于1的M-驻点的数量。因此,从全局优化的角度来看,不能忽略这种鞍点的出现。由于细胞附着的多重性,一个鞍点可能导致两个以上不同的局部极小。我们得出结论,相对复杂的鞍点的结构是众所周知的困难,如果解决SCNO的全局最优性的来源。
英文摘要:
We study sparsity constrained nonlinear optimization (SCNO) from a topological point of view. Special focus will be on M-stationary points from Burdakov et al. (2016). We introduce nondegenerate M-stationary points and define their M-index. We show that all M-stationary points are generically nondegenerate. In particular, the sparsity constraint is active at all local minimizers of a generic SCNO. Some relations to other stationarity concepts, such as S-stationarity, basic feasibility, and CW-minimality, are discussed in detail. By doing so, the issues of instability and degeneracy of points due to different stationarity concepts are highlighted. The concept of M-stationarity allows to adequately describe the global structure of SCNO along the lines of Morse theory. For that, we study topological changes of lower level sets while passing an M-stationary point. As novelty for SCNO, multiple cells of dimension equal to the M-index are needed to be attached. This intriguing fact is in strong contrast with other optimization problems considered before, where just one cell suffices. As a consequence, we derive a Morse relation for SCNO, which relates the numbers of local minimizers and M-stationary points of M-index equal to one. The appearance of such saddle points cannot be thus neglected from the perspective of global optimization. Due to the multiplicity phenomenon in cell-attachment, a saddle point may lead to more than two different local minimizers. We conclude that the relatively involved structure of saddle points is the source of well-known difficulty if solving SCNO to global optimality.
Orthant Based Proximal Stochastic Gradient Method for $\ell_1$-Regularized Optimization
Microsoft(微软);Johns Hopkins University(约翰斯·霍普金斯大学);Zhejiang University(浙江大学);Georgia Institute of Technology(佐治亚理工学院);University of Washington(华盛顿大学);University of Denver(丹佛大学)
Convergence of the Safeguarded Augmented Lagrangian Method under the Polyak-Lojasiewicz constraint qualification for Constrained Composite Optimization