We propose a constrained high-index saddle dynamics (CHiSD) method to search for index-$k$ saddle points of an energy functional subject to equality constraints. With Riemannian manifold tools, the CHiSD is derived in a minimax framework, and its linear stability at an index-$k$ saddle point is proved. To ensure the manifold property, the CHiSD is numerically implemented using retractions and vector transport. Then we present a numerical approach by combining CHiSD with downward and upward search algorithms to construct the solution landscape in the presence of equality constraints. We apply the Thomson problem and the Bose-Einstein condensation as numerical examples to demonstrate the efficiency of the proposed method.
We propose a model-free shrinking-dimer saddle dynamics for finding any-index saddle points and constructing the solution landscapes, in which the force in the standard saddle dynamics is replaced by a surrogate model trained by the Gassian process learning. By this means, the exact form of the model is no longer necessary such that the saddle dynamics could be implemented based only on some observations of the force. This data-driven approach not only avoids the modeling procedure that could be difficult or inaccurate, but also significantly reduces the number of queries of the force that may…
High-index saddle dynamics provides an effective means to compute the any-index saddle points and construct the solution landscape. In this paper we prove error estimates for Euler discretization of high-index saddle dynamics with respect to the time step size, which remains untreated in the literature. We overcome the main difficulties that lie in the strong nonlinearity of the saddle dynamics and the orthonormalization procedure in the numerical scheme that is uncommon in standard discretization of differential equations. The derived methods are further extended to study the generalized high…
Saddle dynamics is a time continuous dynamics to efficiently compute the any-index saddle points and construct the solution landscape. In practice, the saddle dynamics needs to be discretized for numerical computations, while the corresponding numerical analysis are rarely studied in the literature, especially for the high-index cases. In this paper we propose the convergence analysis of discrete high-index saddle dynamics. To be specific, we prove the local linear convergence rates of numerical schemes of high-index saddle dynamics, which indicates that the local curvature in the neighborhood…
We present a mathematical and numerical investigation to the shrinkingdimer saddle dynamics for finding any-index saddle points in the solution landscape. Due to the dimer approximation of Hessian in saddle dynamics, the local Lipschitz assumptions and the strong nonlinearity for the saddle dynamics, it remains challenges for delicate analysis, such as the the boundedness of the solutions and the dimer error. We address these issues to bound the solutions under proper relaxation parameters, based on which we prove the error estimates for numerical discretization to the shrinking-dimer saddle d…
We investigate critical points of the Onsager free-energy model on a sphere with different potential kernels, including the dipolar potential, the Maier-Saupe potential, the coupled dipolar/Maier-Saupe potential, and the Onsager potential. A uniform sampling method is implemented for the discretization of the Onsager model, and solution landscapes of the Onsager model are constructed using saddle dynamics coupled with downward/upward search algorithms. We first construct the solution landscapes with the dipolar and Maier-Saupe potentials, for which all critical points are axisymmetric. For the…
ETH Zurich（苏黎世联邦理工学院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）；University of Münster（明斯特大学）
In this paper, we analyze the landscape of the true loss of neural networks with one hidden layer and ReLU, leaky ReLU, or quadratic activation. In all three cases, we provide a complete classification of the critical points in the case where the target function is affine and one-dimensional. In particular, we show that there exist no local maxima and clarify the structure of saddle points. Moreover, we prove that non-global local minima can only be caused by `dead' ReLU neurons. In particular, they do not appear in the case of leaky ReLU or quadratic activation. Our approach is of a combinato…
