Group synchronization aims to recover the group elements from their noisy pairwise measurements. It has found many applications in community detection, clock synchronization, and joint alignment problem. This paper focuses on the orthogonal group synchronization which is often used in cryo-EM and computer vision. However, it is generally NP-hard to retrieve the group elements by finding the least squares estimator. In this work, we first study the semidefinite programming (SDP) relaxation of the orthogonal group synchronization and its tightness, i.e., the SDP estimator is exactly equal to the least squares estimator. Moreover, we investigate the performance of the Burer-Monteiro factorization in solving the SDP relaxation by analyzing its corresponding optimization landscape. We provide deterministic sufficient conditions which guarantee: (i) the tightness of SDP relaxation; (ii) optimization landscape arising from the Burer-Monteiro approach is benign, i.e., the global optimum is exactly the least squares estimator and no other spurious local optima exist. Our result provides a solid theoretical justification of why the Burer-Monteiro approach is remarkably efficient and effective in solving the large-scale SDPs arising from orthogonal group synchronization. We perform numerical experiments to complement our theoretical analysis, which gives insights into future research directions.
This paper studies an optimization problem on the sum of traces of matrix quadratic forms on $m$ orthogonal matrices, which can be considered as a generalization of the synchronization of rotations. While the problem is nonconvex, the paper shows that its semidefinite programming relaxation can solve the original nonconvex problems exactly, under an additive noise model with small noise in the order of $O(-m^{1/4})$, where $m$ is the number of orthogonal matrices. This result can be considered as a generalization of existing results on phase synchronization.
This paper studies an optimization problem on the sum of traces of matrix quadratic forms in $m$ semi-orthogonal matrices, which can be considered as a generalization of the synchronization of rotations. While the problem is nonconvex, the paper shows that its semidefinite programming relaxation solves the original nonconvex problems exactly with high probability, under an additive noise model with small noise in the order of $O(m^{1/4})$. In addition, it shows that the solution of a nonconvex algorithm considered in Won, Zhou, and Lange [SIAM J. Matrix Anal. Appl., 2 (2021), pp. 859-882] is a…
Given the noisy pairwise measurements among a set of unknown group elements, how to recover them efficiently and robustly? This problem, known as group synchronization, has drawn tremendous attention in the scientific community. In this work, we focus on orthogonal group synchronization that has found many applications, including computer vision, robotics, and cryo-electron microscopy. One commonly used approach is the least squares estimation that requires solving a highly nonconvex optimization program. The past few years have witnessed considerable advances in tackling this challenging prob…
Studying the landscape of nonconvex cost function is key towards a better understanding of optimization algorithms widely used in signal processing, statistics, and machine learning. Meanwhile, the famous Kuramoto model has been an important mathematical model to study the synchronization phenomena of coupled oscillators over various network topologies. In this paper, we bring together these two seemingly unrelated objects by investigating the optimization landscape of a nonlinear function $E(\boldsymbolθ) = \frac{1}{2}\sum_{1\leq i,j\leq n} a_{ij}(1-\cos(θ_i - θ_j))$ associated to an underlyi…
Synchronization on circles and spheres with nonlinear interactions
The Wharton School, University of Pennsylvania, USA（沃森学校，宾夕法尼亚大学）；Institute of Mathematics, EPFL, Lausanne, Switzerland（数学研究所，EPFL，拉沃斯纳，瑞士）；CERMICS, ENPC, Institut Polytechnique de Paris, CNRS, France（CERMICS，ENPC，巴黎理工学院，CNRS，法国）
Peking University（北京大学）；University of Pennsylvania（宾夕法尼亚大学）
Gradient clipping is commonly used in training deep neural networks partly due to its practicability in relieving the exploding gradient problem. Recently, \citet{zhang2019gradient} show that clipped (stochastic) Gradient Descent (GD) converges faster than vanilla GD/SGD via introducing a new assumption called $(L_0, L_1)$-smoothness, which characterizes the violent fluctuation of gradients typically encountered in deep neural networks. However, their iteration complexities on the problem-dependent parameters are rather pessimistic, and theoretical justification of clipping combined with other…
Substantial progress has been made recently on developing provably accurate and efficient algorithms for low-rank matrix factorization via nonconvex optimization. While conventional wisdom often takes a dim view of nonconvex optimization algorithms due to their susceptibility to spurious local minima, simple iterative methods such as gradient descent have been remarkably successful in practice. The theoretical footings, however, had been largely lacking until recently. In this tutorial-style overview, we highlight the important role of statistical models in enabling efficient nonconvex optimiz…
