We study the convergence of the Riemannian steepest descent algorithm on the Grassmann manifold for minimizing the block version of the Rayleigh quotient of a symmetric matrix. Even though this problem is non-convex in the Euclidean sense and only very locally convex in the Riemannian sense, we discover a structure for this problem that is similar to geodesic strong convexity, namely, weak-strong convexity. This allows us to apply similar arguments from convex optimization when studying the convergence of the steepest descent algorithm but with initialization conditions that do not depend on the eigengap $δ$. When $δ>0$, we prove exponential convergence rates, while otherwise the convergence is algebraic. Additionally, we prove that this problem is geodesically convex in a neighbourhood of the global minimizer of radius $\mathcal{O}(\sqrtδ)$.
Convex optimization is a vibrant and successful area due to the existence of a variety of efficient algorithms that leverage the rich structure provided by convexity. Convexity of a smooth set or a function in a Euclidean space is defined by how it interacts with the standard differential structure in this space -- the Hessian of a convex function has to be positive semi-definite everywhere. However, in recent years, there is a growing demand to understand non-convexity and develop computational methods to optimize non-convex functions. Intriguingly, there is a type of non-convexity that disap…
The self regulation problem as an inexact steepest descent method for multicriteria optimization
IME-Universidade Federal de Goiás（戈亚斯联邦大学数学与工程学院）；DM, Universidade Federal do Piauí（皮奥伊联邦大学数学系）；COPPE/Sistemas-Universidade Federal do Rio de Janeiro（里约热内卢联邦大学科佩系统工程研究所）；Aix-Marseille University（艾克斯-马赛大学）
$λ$-geodesic convexity of the squared sliced-Wasserstein distance
Università degli Studi di Padova（帕多瓦大学）；Université Lyon 1, Ecole Centrale de Lyon, INSA Lyon, Université Jean Monnet, CNRS, ICJ, UMR 5208（里昂第一大学，里昂中央理工学院，里昂国立应用科学学院，让·穆兰大学，法国国家科学研究中心，ICJ）
The aim of this note is to compare the averaged optimal coplanar transfer towards circular orbits when the costs are the transfer time transfer and the energy consumption. While the energy case leads to analyze a 2D Riemannian metric using the standard tools of Riemannian geometry (curvature computations, geodesic convexity), the time minimal case is associated to a Finsler metric which is not smooth. Nevertheless a qualitative analysis of the geodesic flow is given in this article to describe the optimal transfers. In particular we prove geodesic convexity of the elliptic domain.
Geodesic convexity generalizes the notion of (vector space) convexity to nonlinear metric spaces. But unlike convex optimization, geodesically convex (g-convex) optimization is much less developed. In this paper we contribute to the understanding of g-convex optimization by developing iteration complexity analysis for several first-order algorithms on Hadamard manifolds. Specifically, we prove upper bounds for the global complexity of deterministic and stochastic (sub)gradient methods for optimizing smooth and nonsmooth g-convex functions, both with and without strong g-convexity. Our analysis…
This paper considers the analysis of continuous time gradient-based optimization algorithms through the lens of nonlinear contraction theory. It demonstrates that in the case of a time-invariant objective, most elementary results on gradient descent based on convexity can be replaced by much more general results based on contraction. In particular, gradient descent converges to a unique equilibrium if its dynamics are contracting in any metric, with convexity of the cost corresponding to the special case of contraction in the identity metric. More broadly, contraction analysis provides new ins…
In this paper, we study classes of discrete convex functions: submodular functions on modular semilattices and L-convex functions on oriented modular graphs. They were introduced by the author in complexity classification of minimum 0-extension problems. We clarify the relationship to other discrete convex functions, such as $k$-submodular functions, skew-bisubmodular functions, L$^\natural$-convex functions, tree submodular functions, and UJ-convex functions. We show that they are actually viewed as submodular/L-convex functions in our sense. We also prove a sharp iteration bound of the steep…
In this paper we consider a new problem of minimizing an M-convex function under L1-distance constraint (MML1); the constraint is given by an upper bound for L1-distance between a feasible solution and a given "center." This is motivated by a nonlinear integer programming problem for re-allocation of dock capacity in a bike sharing system discussed by Freund et al. (2017). The main aim of this paper is to better understand the combinatorial structure of the dock re-allocation problem through the connection with M-convexity, and show its polynomial-time solvability using this connection. For th…
The main goal of this paper is to present results of existence and non-existence of convex functions on Riemannian manifolds and, in the case of the existence, we associate such functions to the geometry of the manifold. Precisely, we prove that the conservativity of the geodesic flow on a Rieman- nain manifold with infinite volume is an obstruction to the existence of convex functions. Next, we present a geometric condition that ensures the existence of (strictly) convex functions on a particular class of complete non-compact man- ifolds, and, we use this fact to construct a manifold whose se…
Harvard University（哈佛大学）；MIT（麻省理工学院）；University of Oxford（牛津大学）
We study geodesically convex (g-convex) problems that can be written as a difference of Euclidean convex functions. This structure arises in several optimization problems in statistics and machine learning, e.g., for matrix scaling, M-estimators for covariances, and Brascamp-Lieb inequalities. Our work offers efficient algorithms that on the one hand exploit g-convexity to ensure global optimality along with guarantees on iteration complexity. On the other hand, the split structure permits us to develop Euclidean Majorization-Minorization algorithms that help us bypass the need to compute expe…
We consider the popular and classical method of alternating projections for finding a point in the intersection of two closed sets. By situating the algorithm in a metric space, equipped only with well-behaved geodesics and angles (in the sense of Alexandrov), we are able to highlight the two key geometric ingredients in a standard intuitive analysis of local linear convergence. The first is a transversality-like condition on the intersection; the second is a convexity-like condition on one set: "uniform approximation by geodesics."
