The Symmetric Tensor Approximation problem (STA) consists of approximating a symmetric tensor or a homogeneous polynomial by a linear combination of symmetric rank-1 tensors or powers of linear forms of low symmetric rank. We present two new Riemannian Newton-type methods for low rank approximation of symmetric tensor with complex coefficients.The first method uses the parametrization of the set of tensors of rank at most $r$ by weights and unit vectors.Exploiting the properties of the apolar product on homogeneous polynomials combined with efficient tools from complex optimization, we provide an explicit and tractable formulation of the Riemannian gradient and Hessian, leading to Newton iterations with local quadratic convergence. We prove that under some regularity conditions on non-defective tensors in the neighborhood of the initial point, the Newton iteration (completed with a trust-region scheme) is converging to a local minimum.The second method is a Riemannian Gauss--Newton method on the Cartesian product of Veronese manifolds. An explicit orthonormal basis of the tangent space of this Riemannian manifold is described. We deduce the Riemannian gradient and the Gauss--Newton approximation of the Riemannian Hessian. We present a new retraction operator on the Veronese manifold.We analyze the numerical behavior of these methods, with an initial point provided by Simultaneous Matrix Diagonalisation (SMD).Numerical experiments show the good numerical behavior of the two methods in different cases and in comparison with existing state-of-the-art methods.
In the present paper, in order to fnd a singularity of a vector field defined on Riemannian manifolds, we present a new globalization strategy of Newton method and establish its global convergence with superlinear rate. In particular, this globalization generalizes for a general retraction the existing damped Newton's method. The presented global convergence analysis does not require any hypotesesis on singularity of the vector field. We applied the proposed method to solve the truncated singular value problem on the product of two Stiefel manifolds, the dextrous hand grasping problem on the c…
This paper is concerned with the inverse problem of constructing a symmetric nonnegative matrix from realizable spectrum. We reformulate the inverse problem as an underdetermined nonlinear matrix equation over a Riemannian product manifold. To solve it, we develop a Riemannian underdetermined inexact Newton dogleg method for solving a general underdetermined nonlinear equation defined between Riemannian manifolds and Euclidean spaces. The global and quadratic convergence of the proposed method is established under some mild assumptions. Then we solve the inverse problem by applying the propose…
Effective Hamiltonians arise in several problems, including homogenization of Hamilton--Jacobi equations, nonlinear control systems, Hamiltonian dynamics, and Aubry--Mather theory. In Aubry--Mather theory, related objects, Mather measures, are also of great importance. Here, we combine ideas from mean-field games with the Hessian Riemannian flow to compute effective Hamiltonians and Mather measures simultaneously. We prove the convergence of the Hessian Riemannian flow in the continuous setting. For the discrete case, we give both the existence and the convergence of the Hessian Riemannian flo…
The University of Electro-Communications（电气通信大学）；Tokyo University of Science（东京理科大学）
Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite number of loss functions. The present paper proposes a Riemannian stochastic quasi-Newton algorithm with variance reduction (R-SQN-VR). The key challenges of averaging, adding, and subtracting multiple gradients are addressed with notions of retraction and vector transport. We present convergence analyses of R-SQN-VR on both non-convex and retraction-convex functions under retraction and vector transport operators. The proposed algorithm is evaluated on the Karcher mean compu…
In this paper, we consider the inverse eigenvalue problem for the positive doubly stochastic matrices, which aims to construct a positive doubly stochastic matrix from the prescribed realizable spectral data. By using the real Schur decomposition, the inverse problem is written as a nonlinear matrix equation on a matrix product manifold. We propose monotone and nonmonotone Riemannian inexact Newton-CG methods for solving the nonlinear matrix equation. The global and quadratic convergence of the proposed methods is established under some assumptions. We also provide invariant subspaces of the c…
The short-time Fourier transform (STFT), or the discrete Gabor transform (DGT), has been extensively used in signal analysis and processing. Their properties are characterized by a window function. For signal processing, designing a special window called tight window is important because it is known to make DGT-domain processing robust to error. In this paper, we propose a method of designing tight windows that minimize the sidelobe energy. It is formulated as a constrained spectral concentration problem, and a Newton's method on an oblique manifold is derived to efficiently obtain a solution.…
We consider the low-rank tensor train completion problem when additional side information is available in the form of subspaces that contain the mode-$k$ fiber spans. We propose an algorithm based on Riemannian optimization to solve the problem. Numerical experiments show that the proposed algorithm requires far fewer known entries to recover the tensor compared to standard tensor train completion methods.
Marchuk Institute of Numerical Mathematics of the Russian Academy of Sciences（俄罗斯科学院马尔丘克计算数学研究所）；HSE University（高等经济学院）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术学院）
In scientific computing and machine learning applications, matrices and more general multidimensional arrays (tensors) can often be approximated with the help of low-rank decompositions. Since matrices and tensors of fixed rank form smooth Riemannian manifolds, one of the popular tools for finding low-rank approximations is to use Riemannian optimization. Nevertheless, efficient implementation of Riemannian gradients and Hessians, required in Riemannian optimization algorithms, can be a nontrivial task in practice. Moreover, in some cases, analytic formulas are not even available. In this pape…
We propose new Riemannian preconditioned algorithms for low-rank tensor completion via the polyadic decomposition of a tensor. These algorithms exploit a non-Euclidean metric on the product space of the factor matrices of the low-rank tensor in the polyadic decomposition form. This new metric is designed using an approximation of the diagonal blocks of the Hessian of the tensor completion cost function, thus has a preconditioning effect on these algorithms. We prove that the proposed Riemannian gradient descent algorithm globally converges to a stationary point of the tensor completion problem…
We propose a new modification of Newton iteration for finding some nonnegative Z-eigenpairs of a nonnegative tensor. The method has local quadratic convergence to a nonnegative eigenpair of a nonnegative tensor, under the usual assumption guaranteeing the local quadratic convergence of the original Newton iteration.
Natural Riemannian gradient for learning functional tensor networks
Institute of Mathematics, University of Augsburg（奥格斯堡大学数学研究所）；Department of Mathematics, Technical University of Munich（慕尼黑工业大学数学系）；Institute of Mathematics & Centre for Advanced Analytics and Predictive Sciences, University of Augsburg（奥格斯堡大学数学研究所与高级分析与预测科学中心）
Tensor factorizations with nonnegative constraints have found application in analyzing data from cyber traffic, social networks, and other areas. We consider application data best described as being generated by a Poisson process (e.g., count data), which leads to sparse tensors that can be modeled by sparse factor matrices. In this paper we investigate efficient techniques for computing an appropriate canonical polyadic tensor factorization based on the Kullback-Leibler divergence function. We propose novel subproblem solvers within the standard alternating block variable approach. Our new me…
