Improved Penalty Function Approaches for Optimization Problems with General Orthogonality
Guangxi University（广西大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Chinese Academy of Sciences（中国科学院大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
In this paper, we consider a class of generalized orthogonal optimization constraint problems (GOOCP) over $\mathbb{R}^{n \times p}$, where the variable $X$ is restricted within the intersection of a certain subspace $\mathcal{F}$ and satisfies the quadratic constraint $\{X \in \mathbb{R}^{n \times p}: X^{\top} ϕ(X) = I_p\}$. Such constraints generalize a wide range of structured matrix manifolds, such as the Stiefel manifold, the symplectic Stiefel manifold, the indefinite Stiefel manifold, the third-order tensor Stiefel manifold, etc. We show that the feasible region of GOOCP is a closed embedded submanifold of $\mathbb{R}^{n \times p}$ and characterize the necessary geometric materials for the existing Riemannian optimization frameworks. Based on the constraint dissolving approach for Riemannian optimization problems, we propose the constraint dissolving penalty function (GOCDF) for the constrained optimization problem GOOCP with easy-to-compute formulations. We further establish the equivalence between GOCDF and GOOCP in the aspects of first-order and second-order stationary points. We also analyze the computational complexity of applying first-order methods to minimize GOOCP, which could be significantly lower than those of first-order Riemannian optimization methods. Numerical experiments demonstrate that solving GOOCP through applying unconstrained optimization methods to minimize constraint dissolving function demonstrates superior efficiency to existing Riemannian optimization methods.
Science and Technology Centre Novo mesto（新梅斯托科学与技术中心）；Faculty of mathematics and physics, University of Ljubljana（卢布尔雅那大学数学物理学院）；Faculty of mechanical engineering, University of Ljubljana（卢布尔雅那大学机械工程学院）；Department of Mathematics, University of Klagenfurt（克拉根福大学数学系）
MoSSP: A Momentum-Based Single-Loop Stochastic Penalty Method for Nonconvex Constrained DC-Regularized Optimization
School of Mathematical Sciences, Beihang University, Beijing 100191, China（北京航空航天大学数学学院）；School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou 510006, China（中山大学计算机科学与工程学院）
