Unconstrained optimisation on Riemannian manifolds
Department of Mathematics, University of Oslo（奥斯陆大学数学系）
Tuyen Trung Truong
备注：
29 pages. Some experimental results (on singular cost functions on Euclidean spaces, and on minimum on the closed unit ball in the Euclidean spaces) are given. References updated
In this paper, we give explicit descriptions of versions of (Local-) Backtracking Gradient Descent and New Q-Newton's method to the Riemannian setting.Here are some easy to state consequences of results in this paper, where X is a general Riemannian manifold of finite dimension and $f:X\rightarrow \mathbb{R}$ a $C^2$ function which is Morse (that is, all its critical points are non-degenerate).
{\bf Theorem.} For random choices of the hyperparameters in the Riemanian Local Backtracking Gradient Descent algorithm and for random choices of the initial point $x_0$, the sequence $\{x_n\}$ constructed by the algorithm either (i) converges to a local minimum of $f$ or (ii) eventually leaves every compact subsets of $X$ (in other words, diverges to infinity on $X$). If $f$ has compact sublevels, then only the former alternative happens. The convergence rate is the same as in the classical paper by Armijo.
{\bf Theorem.} Assume that $f$ is $C^3$. For random choices of the hyperparametes in the Riemannian New Q-Newton's method, if the sequence constructed by the algorithm converges, then the limit is a critical point of $f$. We have a local Stable-Center manifold theorem, near saddle points of $f$, for the dynamical system associated to the algorithm. If the limit point is a non-degenerate minimum point, then the rate of convergence is quadratic. If moreover $X$ is an open subset of a Lie group and the initial point $x_0$ is chosen randomly, then we can globally avoid saddle points.
As an application, we propose a general method using Riemannian Backtracking GD to find minimum of a function on a bounded ball in a Euclidean space, and do explicit calculations for calculating the smallest eigenvalue of a symmetric square matrix.
Department of Computational Applied Mathematics；Operations Research, Rice University, Houston, TX, USA；Department of Statistics, Texas A\&M University, College Station, TX, USA
Min-Max Optimisation for Nonconvex-Nonconcave Functions Using a Random Zeroth-Order Extragradient Algorithm
School of Engineering Australian National University（工程学院澳大利亚国立大学）；CIICADA Lab（CIICADA实验室）；Department of Electrical Engineering Polytechnique Montréal（电气工程系蒙特利尔理工学院）；GERAD & Mila（GERAD与Mila）；Department of Mathematics and Industrial Engineering Polytechnique Montréal（数学与工业工程系蒙特利尔理工学院）；GERAD；Department of Electrical and Electronic Engineering University of Melbourne（电气与电子工程系墨尔本大学）
On the convergence of orthogonalization-free conjugate gradient method for extreme eigenvalues of Hermitian matrices: a Riemannian optimization interpretation
A Second-Order Algorithm Based on Affine Scaling Interior-Point Methods for nonlinear Optimisation with bound constraints
School of Mathematics and Statistics, Henan Normal University（河南师范大学数学与统计学院）；Instituto de Matemática e Estatística, Universidade Federal de Goiás（戈亚斯联邦大学数学与统计研究所）；Mathematics and Science College, Shanghai Normal University（上海师范大学数理学院）
A primal-dual interior point trust region method for second-order stationary points of Riemannian inequality-constrained optimization problems
Graduate School of Information Science and Technology, The University of Tokyo（东京大学大学院信息理工学系研究科）；Center for Advanced Intelligence Project, RIKEN（理化学研究所先进智能项目中心）；Faculty of Science and Technology, Seikei University（立教大学理学部）
Federated Learning on Riemannian Manifolds: A Gradient-Free Projection-Based Approach
School of Information Management and Engineering, Shanghai University of Finance and Economics（上海金融学院信息管理与工程学院）；School of Information Management and Engineering, Shanghai University of Finance and Economics（上海金融学院信息管理与工程学院；明尼苏达大学工业与系统工程系）；Department of Industrial and System Engineering, University of Minnesota（明尼苏达大学电气与计算机工程系）；Department of Electrical and Computer Engineering, University of Minnesota
Improved Penalty Function Approaches for Optimization Problems with General Orthogonality
Guangxi University（广西大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Chinese Academy of Sciences（中国科学院大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
