我们获得了Riemannian Gauss-Newton方法在流形上求解最小二乘问题的局部收敛结果中出现的乘法常数的估计，并将其与[P. Bürgisser和F. Cucker, Condition: The Geometry of Numerical Algorithms, 2013]中的几何条件数联系起来。
英文摘要：
We obtain estimates of the multiplicative constants appearing in local convergence results of the Riemannian Gauss-Newton method for least squares problems on manifolds and relate them to the geometric condition number of [P. Bürgisser and F. Cucker, Condition: The Geometry of Numerical Algorithms, 2013].
This paper is concerned with the inverse problem of constructing a symmetric nonnegative matrix from realizable spectrum. We reformulate the inverse problem as an underdetermined nonlinear matrix equation over a Riemannian product manifold. To solve it, we develop a Riemannian underdetermined inexact Newton dogleg method for solving a general underdetermined nonlinear equation defined between Riemannian manifolds and Euclidean spaces. The global and quadratic convergence of the proposed method is established under some mild assumptions. Then we solve the inverse problem by applying the propose…
Effective Hamiltonians arise in several problems, including homogenization of Hamilton--Jacobi equations, nonlinear control systems, Hamiltonian dynamics, and Aubry--Mather theory. In Aubry--Mather theory, related objects, Mather measures, are also of great importance. Here, we combine ideas from mean-field games with the Hessian Riemannian flow to compute effective Hamiltonians and Mather measures simultaneously. We prove the convergence of the Hessian Riemannian flow in the continuous setting. For the discrete case, we give both the existence and the convergence of the Hessian Riemannian flo…
In this paper, we consider the inverse eigenvalue problem for the positive doubly stochastic matrices, which aims to construct a positive doubly stochastic matrix from the prescribed realizable spectral data. By using the real Schur decomposition, the inverse problem is written as a nonlinear matrix equation on a matrix product manifold. We propose monotone and nonmonotone Riemannian inexact Newton-CG methods for solving the nonlinear matrix equation. The global and quadratic convergence of the proposed methods is established under some assumptions. We also provide invariant subspaces of the c…
The Symmetric Tensor Approximation problem (STA) consists of approximating a symmetric tensor or a homogeneous polynomial by a linear combination of symmetric rank-1 tensors or powers of linear forms of low symmetric rank. We present two new Riemannian Newton-type methods for low rank approximation of symmetric tensor with complex coefficients.The first method uses the parametrization of the set of tensors of rank at most $r$ by weights and unit vectors.Exploiting the properties of the apolar product on homogeneous polynomials combined with efficient tools from complex optimization, we provide…
In the present paper, in order to fnd a singularity of a vector field defined on Riemannian manifolds, we present a new globalization strategy of Newton method and establish its global convergence with superlinear rate. In particular, this globalization generalizes for a general retraction the existing damped Newton's method. The presented global convergence analysis does not require any hypotesesis on singularity of the vector field. We applied the proposed method to solve the truncated singular value problem on the product of two Stiefel manifolds, the dextrous hand grasping problem on the c…
The University of Electro-Communications（电气通信大学）；Tokyo University of Science（东京理科大学）
Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite number of loss functions. The present paper proposes a Riemannian stochastic quasi-Newton algorithm with variance reduction (R-SQN-VR). The key challenges of averaging, adding, and subtracting multiple gradients are addressed with notions of retraction and vector transport. We present convergence analyses of R-SQN-VR on both non-convex and retraction-convex functions under retraction and vector transport operators. The proposed algorithm is evaluated on the Karcher mean compu…
The short-time Fourier transform (STFT), or the discrete Gabor transform (DGT), has been extensively used in signal analysis and processing. Their properties are characterized by a window function. For signal processing, designing a special window called tight window is important because it is known to make DGT-domain processing robust to error. In this paper, we propose a method of designing tight windows that minimize the sidelobe energy. It is formulated as a constrained spectral concentration problem, and a Newton's method on an oblique manifold is derived to efficiently obtain a solution.…
Phase recovery from the bispectrum is a central problem in speckle interferometry which can be posed as an optimization problem minimizing a weighted nonlinear least-squares objective function. We look at two different formulations of the phase recovery problem from the literature, both of which can be minimized with respect to either the recovered phase or the recovered image. Previously, strategies for solving these formulations have been limited to first-order optimization methods such as gradient descent or quasi-Newton methods. This paper explores Gauss-Newton optimization schemes for the…
When a physical system is modeled by a nonlinear function, the unknown parameters can be estimated by fitting experimental observations by a least-squares approach. Newton's method and its variants are often used to solve problems of this type. In this paper, we are concerned with the computation of the minimal-norm solution of an underdetermined nonlinear least-squares problem. We present a Gauss-Newton type method, which relies on two relaxation parameters to ensure convergence, and which incorporates a procedure to dynamically estimate the two parameters, as well as the rank of the Jacobian…
In this paper we consider the iteratively regularized Gauss-Newton method, where regularization is achieved by Ivanov regularization, i.e., by imposing a priori constraints on the solution. We propose an a posteriori choice of the regularization radius, based on an inexact Newton / discrepancy principle approach, prove convergence and convergence rates under a variational source condition as the noise level tends to zero, and provide an analysis of the discretization error. Our results are valid in general, possibly nonreflexive Banach spaces, including, e.g., $L^\infty$ as a preimage space. T…
We present a sparse Gauss-Newton solver for accelerated sensitivity analysis with applications to a wide range of equilibrium-constrained optimization problems. Dense Gauss-Newton solvers have shown promising convergence rates for inverse problems, but the cost of assembling and factorizing the associated matrices has so far been a major stumbling block. In this work, we show how the dense Gauss-Newton Hessian can be transformed into an equivalent sparse matrix that can be assembled and factorized much more efficiently. This leads to drastically reduced computation times for many inverse probl…
As second-order methods, Gauss--Newton-type methods can be more effective than first-order methods for the solution of nonsmooth optimization problems with expensive-to-evaluate smooth components. Such methods, however, often do not converge. Motivated by nonlinear inverse problems with nonsmooth regularization, we propose a new Gauss--Newton-type method with inexact relaxed steps. We prove that the method converges to a set of disjoint critical points given that the linearisation of the forward operator for the inverse problem is sufficiently precise. We extensively evaluate the performance o…
Alternating least squares is the most widely used algorithm for CP tensor decomposition. However, alternating least squares may exhibit slow or no convergence, especially when high accuracy is required. An alternative approach is to regard CP decomposition as a nonlinear least squares problem and employ Newton-like methods. Direct solution of linear systems involving an approximated Hessian is generally expensive. However, recent advancements have shown that use of an implicit representation of the linear system makes these methods competitive with alternating least squares. We provide the fir…
Domain decomposition architectures and Gauss-Newton training for physics-informed neural networks
Alexander Heinlein Delft Institute of Applied Mathematics, Delft University of Technology, Mekelweg 4, Delft, 2628 CD, the Netherlands, Taniya Kapoor Artificial Intelligence Group, Wageningen University \& Research, Wageningen, The Netherlands
