This paper presents new Riemannian conjugate gradient methods and global convergence analyses under the strong Wolfe conditions. The main idea of the new methods is to combine the good global convergence properties of the Dai-Yuan method with the efficient numerical performance of the Hestenes-Stiefel method. The proposed methods compare well numerically with the existing methods for the Rayleigh quotient minimization problem on the unit sphere. Numerical comparisons show that they perform better than the existing ones.
The left-invariant sub-Riemannian problem on the Engel group is considered. This problem is very important as nilpotent approximation of nonholonomic systems in four-dimensional space with two-dimensional control, for instance of a system which describes movement of mobile trailer robot. We study the local optimality of extremal trajectories and estimate conjugate time in this article.
This article describes a new Riemannian conjugate gradient method and presents a global convergence analysis. The existing Fletcher-Reeves-type Riemannian conjugate gradient method is guaranteed to be globally convergent if it is implemented with the strong Wolfe conditions. On the other hand, the Dai-Yuan-type Euclidean conjugate gradient method generates globally convergent sequences under the weak Wolfe conditions. This article deals with a generalization of Dai-Yuan's Euclidean algorithm to a Riemannian algorithm that requires only the weak Wolfe conditions. The global convergence property…
This paper proposes a novel general framework of Riemannian conjugate gradient methods, that is, conjugate gradient methods on Riemannian manifolds. The conjugate gradient methods are important first-order optimization algorithms both in Euclidean spaces and on Riemannian manifolds. While various types of conjugate gradient methods are studied in Euclidean spaces, there have been fewer studies on those on Riemannian manifolds. In each iteration of the Riemannian conjugate gradient methods, the previous search direction must be transported to the current tangent space so that it can be added to…
This paper considers sufficient descent Riemannian conjugate gradient methods with line search algorithms. We propose two kinds of sufficient descent nonlinear conjugate gradient methods and prove these methods satisfy the sufficient descent condition even on Riemannian manifolds. One is the hybrid method combining the Fletcher-Reeves-type method with the Polak-Ribiere-Polyak-type method, and the other is the Hager-Zhang-type method, both of which are generalizations of those used in Euclidean space. Also, we generalize two kinds of line search algorithms that are widely used in Euclidean spac…
This paper presents the Hager-Zhang (HZ)-type Riemannian conjugate gradient method that uses the exponential retraction. We also present global convergence analyses of our proposed method under two kinds of assumptions. Moreover, we numerically compare our proposed methods with the existing methods by solving two kinds of Riemannian optimization problems on the unit sphere. The numerical results show that our proposed method has much better performance than the existing methods, i.e., the FR, DY, PRP and HS methods. In particular, they show that it has much higher performance than existing met…
This article deals with the conjugate gradient method on a Riemannian manifold with interest in global convergence analysis. The existing conjugate gradient algorithms on a manifold endowed with a vector transport need the assumption that the vector transport does not increase the norm of tangent vectors, in order to confirm that generated sequences have a global convergence property. In this article, the notion of a scaled vector transport is introduced to improve the algorithm so that the generated sequences may have a global convergence property under a relaxed assumption. In the proposed a…
In this paper, we extend and complete the classification of the generic singularities of the 3D-contact sub-Riemmanian conjugate locus in a neighbourhood of the origin.
The goal of tensor completion is to fill in missing entries of a partially known tensor under a low-rank constraint. In this paper, we mainly study low rank third-order tensor completion problems by using Riemannian optimization methods on the smooth manifold. Here the tensor rank is defined to be a set of matrix ranks where the matrices are the slices of the transformed tensor obtained by applying the Fourier-related transformation onto the tubes of the original tensor. We show that with suitable incoherence conditions on the underlying low rank tensor, the proposed Riemannian optimization me…
We study local and global optimality of geodesics in the left invariant sub-Riemannian problem on the Lie group $\mathrm{SH}(2)$. We obtain the complete description of the Maxwell points corresponding to the discrete symmetries of the vertical subsystem of the Hamiltonian system. An effective upper bound on the cut time is obtained in terms of the first Maxwell times. We study the local optimality of extremal trajectories and prove the lower and upper bounds on the first conjugate times. We also obtain the generic time interval for the $n$-th conjugate time which is important in the study of s…
We prove sectional and Ricci-type comparison theorems for the existence of conjugate points along sub-Riemannian geodesics. In order to do that, we regard sub-Riemannian structures as a special kind of variational problems. In this setting, we identify a class of models, namely linear quadratic optimal control systems, that play the role of the constant curvature spaces. As an application, we prove a version of sub-Riemannian Bonnet-Myers theorem and we obtain some new results on conjugate points for three dimensional left-invariant sub-Riemannian structures.
The Cartan group is the free nilpotent Lie group of rank 2 and step 3. We consider the left-invariant sub-Riemannian problem on the Cartan group defined by an inner product in the first layer of its Lie algebra. This problem gives a nilpotent approximation of an arbitrary sub-Riemannian problem with the growth vector $(2,3,5)$. In previous works we described a group of symmetries of the sub-Riemannian problem on the Cartan group, and the corresponding Maxwell time -- the first time when symmetric geodesics intersect one another. It is known that geodesics are not globally optimal after the Max…
This paper contains material originally presented in the first submission of the paper "On the Optimal Control of Impulsive Hybrid Systems On Riemannian Manifolds" to SIAM Journal on Control and Optimization on the 28th of February, 2012. In the revised version of the paper the following sections are removed.
Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold
Zhejiang University（浙江大学）；SGIT AI Lab, State Grid Corporation of China（国网公司SGIT人工智能实验室）；CFAR and IHPC, Agency for Science, Technology and Research（新加坡科技研究局CFAR与IHPC）；SCSE, NTU（南洋理工大学计算机科学与工程学院）
This paper provides a geometrical derivation of the Hybrid Minimum Principle (HMP) for autonomous impulsive hybrid systems on Riemannian manifolds, i.e. systems where the manifold valued component of the hybrid state trajectory may have a jump discontinuity when the discrete component changes value. The analysis is expressed in terms of extremal trajectories on the cotangent bundle of the manifold state space. In the case of autonomous hybrid systems, switching manifolds are defined as smooth embedded submanifolds of the state manifold and the jump function is defined as a smooth map on the sw…
On the convergence of orthogonalization-free conjugate gradient method for extreme eigenvalues of Hermitian matrices: a Riemannian optimization interpretation
In recent years, Riemannian stochastic gradient descent (R-SGD), Riemannian stochastic variance reduction (R-SVRG) and Riemannian stochastic recursive gradient (R-SRG) have attracted considerable attention on Riemannian optimization. Under normal circumstances, it is impossible to analyze the convergence of R-SRG algorithm alone. The main reason is that the conditional expectation of the descending direction is a biased estimation. However, in this paper, we consider linear combination of three descent directions on Riemannian manifolds as the new descent direction (i.e., R-SRG, R-SVRG and R-S…
A solution to the left-invariant sub-Riemannian problem on the group of motions (rototranslations) of a plane SE(2) is obtained. Local and global optimality of extremal trajectories is characterized. Lower and upper bounds on the first conjugate time are proved. The cut time is shown to be equal to the first Maxwell time corresponding to the group of discrete symmetries of the exponential mapping. An explicit global description of the cut locus is obtained. Optimal synthesis is described.
We introduce a generalized Rayleigh-quotient on the tensor product of Grassmannians enabling a unified approach to well-known optimization tasks from different areas of numerical linear algebra, such as best low-rank approximations of tensors (data compression), geometric measures of entanglement (quantum computing) and subspace clustering (image processing). We briefly discuss the geometry of the constraint set, we compute the Riemannian gradient of the generalized Rayleigh-quotient, we characterize its critical points and prove that they are generically non-degenerated. Moreover, we derive a…
In this paper we combine concepts from Riemannian Optimization and the theory of Sobolev gradients to derive a new conjugate gradient method for direct minimization of the Gross-Pitaevskii energy functional with rotation. The conservation of the number of particles constrains the minimizers to lie on a manifold corresponding to the unit $L^2$ norm. The idea developed here is to transform the original constrained optimization problem to an unconstrained problem on this (spherical) Riemannian manifold, so that fast minimization algorithms can be applied as alternatives to more standard constrain…
Peking University（北京大学）；Massachusetts Institute of Technology（麻省理工学院）
We contribute to advancing the understanding of Riemannian accelerated gradient methods. In particular, we revisit Accelerated Hybrid Proximal Extragradient(A-HPE), a powerful framework for obtaining Euclidean accelerated methods \citep{monteiro2013accelerated}. Building on A-HPE, we then propose and analyze Riemannian A-HPE. The core of our analysis consists of two key components: (i) a set of new insights into Euclidean A-HPE itself; and (ii) a careful control of metric distortion caused by Riemannian geometry. We illustrate our framework by obtaining a few existing and new Riemannian accele…
