Low rank matrix and tensor completion problems are to recover the incomplete two and higher order data by using their low rank structures. The essential problem in the matrix and tensor completion problems is how to improve the efficiency. To this end, we first establish the relationship between matrix rank and tensor tubal rank, and then reformulate matrix completion problem as a tensor completion problem. For the reformulated tensor completion problem, we adopt a two-stage strategy based on tensor factorization algorithm. In this way, a matrix completion problem of big size can be solved via some matrix computations of smaller sizes. For a third order tensor completion problem, to fully exploit the low rank structures, we introduce the double tubal rank which combines the tubal rank and the rank of the mode-3 unfolding matrix. For the mode-3 unfolding matrix rank, we follow the idea of matrix completion. Based on this, we establish a novel model and modify the tensor factorization based algorithm for third order tensor completion. Extensive numerical experiments demonstrate that the proposed methods outperform state-of-the-art methods in terms of both accuracy and running time.
Solving semidefinite programs (SDP) in a short time is the key to managing various mathematical optimization problems. The matrix-completion primal-dual interior-point method (MC-PDIPM) extracts a sparse structure of input SDP by factorizing the variable matrices. In this paper, we propose a new factorization based on the inverse of the variable matrix to enhance the performance of MC-PDIPM. We also use multithreaded parallel computing to deal with the major bottlenecks in MC-PDIPM. Numerical results show that the new factorization and multithreaded computing reduce the computation time for SD…
In this paper, we propose a new approach to solve low-rank tensor completion and robust tensor PCA. Our approach is based on some novel notion of (even-order) tensor ranks, to be called the M-rank, the symmetric M-rank, and the strongly symmetric M-rank. We discuss the connections between these new tensor ranks and the CP-rank and the symmetric CP-rank of an even-order tensor. We show that the M-rank provides a reliable and easy-computable approximation to the CP-rank. As a result, we propose to replace the CP-rank by the M-rank in the low-CP-rank tensor completion and robust tensor PCA. Numer…
The low-rank matrix completion problem can be solved by Riemannian optimization on a fixed-rank manifold. However, a drawback of the known approaches is that the rank parameter has to be fixed a priori. In this paper, we consider the optimization problem on the set of bounded-rank matrices. We propose a Riemannian rank-adaptive method, which consists of fixed-rank optimization, rank increase step and rank reduction step. We explore its performance applied to the low-rank matrix completion problem. Numerical experiments on synthetic and real-world datasets illustrate that the proposed rank-adap…
In this paper we study the low rank matrix completion problem using tools from Schur complement. We give a sufficient and necessary condition such that the completed matrix is globally unique with given data. We assume the observed entries of the matrix follow a special "staircase" structure. Under this assumption, the matrix completion problem is either globally unique or has infinitely many solutions (thus excluding local uniqueness). In fact, the uniqueness of the matrix completion problem totally depends on the rank of the submatrices at the corners of the "staircase". The proof of the the…
It is well-known that the Burer-Monteiro (B-M) factorization approach can efficiently solve low-rank matrix optimization problems under the RIP condition. It is natural to ask whether B-M factorization-based methods can succeed on any low-rank matrix optimization problems with a low information-theoretic complexity, i.e., polynomial-time solvable problems that have a unique solution. In this work, we provide a negative answer to the above question. We investigate the landscape of B-M factorized polynomial-time solvable matrix completion (MC) problems, which are the most popular subclass of low…
We use techniques from (tracial noncommutative) polynomial optimization to formulate hierarchies of semidefinite programming lower bounds on matrix factorization ranks. In particular, we consider the nonnegative rank, the positive semidefinite rank, and their symmetric analogues: the completely positive rank and the completely positive semidefinite rank. We study the convergence properties of our hierarchies, compare them extensively to known lower bounds, and provide some (numerical) examples.
In light of recent data science trends, new interest has fallen in alternative matrix factorizations. By this, we mean various ways of factorizing particular data matrices so that the factors have special properties and reveal insights into the original data. We are interested in the specialized ranks associated with these factorizations, but they are usually difficult to compute. In particular, we consider the nonnegative-, completely positive-, and separable ranks. We focus on a general tool for approximating factorization ranks, the moment hierarchy, a classical technique from polynomial op…
Two common approaches in low-rank optimization problems are either working directly with a rank constraint on the matrix variable, or optimizing over a low-rank factorization so that the rank constraint is implicitly ensured. In this paper, we study the natural connection between the rank-constrained and factorized approaches. We show that all second-order stationary points of the factorized objective function correspond to fixed points of projected gradient descent run on the original problem (where the projection step enforces the rank constraint). This result allows us to unify many existin…
Given underdetermined measurements of a Positive Semi-Definite (PSD) matrix $X$ of known low rank $K$, we present a new algorithm to estimate $X$ based on recent advances in non-convex optimization schemes. We apply this in particular to the phase retrieval problem for Fourier data, which can be formulated as a rank 1 PSD matrix recovery problem. Moreover, we provide theory for how oversampling affects the stability of the lifted inverse problem.
The goal of tensor completion is to fill in missing entries of a partially known tensor under a low-rank constraint. In this paper, we mainly study low rank third-order tensor completion problems by using Riemannian optimization methods on the smooth manifold. Here the tensor rank is defined to be a set of matrix ranks where the matrices are the slices of the transformed tensor obtained by applying the Fourier-related transformation onto the tubes of the original tensor. We show that with suitable incoherence conditions on the underlying low rank tensor, the proposed Riemannian optimization me…
In this paper, nonconvex and nonsmooth models for compressed sensing (CS) and low rank matrix completion (MC) is studied. The problem is formulated as a nonconvex regularized leat square optimization problems, in which the l0-norm and the rank function are replaced by l1-norm and nuclear norm, and adding a nonconvex penalty function respectively. An alternating minimization scheme is developed, and the existence of a subsequence, which generate by the alternating algorithm that converges to a critical point, is proved. The NSP, RIP, and RIP condition for stable recovery guarantees also be anal…
We provide the first positive result on the nonsmooth optimization landscape of robust principal component analysis, to the best of our knowledge. It is the object of several conjectures and remains mostly uncharted territory. We identify a necessary and sufficient condition for the absence of spurious local minima in the rank-one case. Our proof exploits the subdifferential regularity of the objective function in order to eliminate the existence quantifier from the first-order optimality condition known as Fermat's rule.
Factorization-based gradient descent is a scalable and efficient algorithm for solving low-rank matrix completion. Recent progress in structured non-convex optimization has offered global convergence guarantees for gradient descent under certain statistical assumptions on the low-rank matrix and the sampling set. However, while the theory suggests gradient descent enjoys fast linear convergence to a global solution of the problem, the universal nature of the bounding technique prevents it from obtaining an accurate estimate of the rate of convergence. In this paper, we perform a local analysis…
University of Liège（列日大学）；University of Cambridge（剑桥大学）
We exploit the versatile framework of Riemannian optimization on quotient manifolds to develop R3MC, a nonlinear conjugate-gradient method for low-rank matrix completion. The underlying search space of fixed-rank matrices is endowed with a novel Riemannian metric that is tailored to the least-squares cost. Numerical comparisons suggest that R3MC robustly outperforms state-of-the-art algorithms across different problem instances, especially those that combine scarcely sampled and ill-conditioned data.
Rank minimization is of interest in machine learning applications such as recommender systems and robust principal component analysis. Minimizing the convex relaxation to the rank minimization problem, the nuclear norm, is an effective technique to solve the problem with strong performance guarantees. However, nonconvex relaxations have less estimation bias than the nuclear norm and can more accurately reduce the effect of noise on the measurements. We develop efficient algorithms based on iteratively reweighted nuclear norm schemes, while also utilizing the low rank factorization for semidefi…
We consider the multi-objective optimization problem of choosing the bottom left block-entry of a block lower triangular matrix to minimize the ranks of all block sub-matrices. We provide a proof that there exists a simultaneous rank-minimizer by constructing the complete set of all minimizers.
Matrix completion is about recovering a matrix from its partial revealed entries, and it can often be achieved by exploiting the inherent simplicity or low dimensional structure of the target matrix. For instance, a typical notion of matrix simplicity is low rank. In this paper we study matrix completion based on another low dimensional structure, namely the low rank Hankel structure in the Fourier domain. It is shown that matrices with this structure can be exactly recovered by solving a convex optimization program provided the sampling complexity is nearly optimal. Empirical results are also…
This paper is concerned with the column $\ell_{2,0}$-regularized factorization model of low-rank matrix recovery problems and its computation. The column $\ell_{2,0}$-norm of factor matrices is introduced to promote column sparsity of factors and low-rank solutions. For this nonconvex discontinuous optimization problem, we develop an alternating majorization-minimization (AMM) method with extrapolation, and a hybrid AMM in which a majorized alternating proximal method is proposed to seek an initial factor pair with less nonzero columns and the AMM with extrapolation is then employed to minimiz…
Hangzhou Dianzi University（杭州电子科技大学）；Ningbo University（宁波大学）；Huawei Technologies Co., Ltd.（华为技术有限公司）；The Hong Kong Polytechnic University（香港理工大学）
Recovering color images and videos from highly undersampled data is a fundamental and challenging task in face recognition and computer vision. By the multi-dimensional nature of color images and videos, in this paper, we propose a novel tensor completion approach, which is able to efficiently explore the sparsity of tensor data under the discrete cosine transform (DCT). Specifically, we introduce two ``sparse + low-rank'' tensor completion models as well as two implementable algorithms for finding their solutions. The first one is a DCT-based sparse plus weighted nuclear norm induced low-rank…
This paper addresses the problem of low-rank distance matrix completion. This problem amounts to recover the missing entries of a distance matrix when the dimension of the data embedding space is possibly unknown but small compared to the number of considered data points. The focus is on high-dimensional problems. We recast the considered problem into an optimization problem over the set of low-rank positive semidefinite matrices and propose two efficient algorithms for low-rank distance matrix completion. In addition, we propose a strategy to determine the dimension of the embedding space. Th…
This paper is concerned with the factorization form of the rank regularized loss minimization problem. To cater for the scenario in which only a coarse estimation is available for the rank of the true matrix, an $\ell_{2,0}$-norm regularized term is added to the factored loss function to reduce the rank adaptively; and account for the ambiguities in the factorization, a balanced term is then introduced. For the least squares loss, under a restricted condition number assumption on the sampling operator, we establish the KL property of exponent $1/2$ of the nonsmooth factored composite function …
We study a non-convex low-rank promoting penalty function, the transformed Schatten-1 (TS1), and its applications in matrix completion. The TS1 penalty, as a matrix quasi-norm defined on its singular values, interpolates the rank and the nuclear norm through a nonnegative parameter a. We consider the unconstrained TS1 regularized low-rank matrix recovery problem and develop a fixed point representation for its global minimizer. The TS1 thresholding functions are in closed analytical form for all parameter values. The TS1 threshold values differ in subcritical (supercritical) parameter regime w…
Hangzhou Dianzi University（杭州电子科技大学）；The Hong Kong Polytechnic University（香港理工大学）；Huawei（华为技术有限公司）
Tensor train (TT) factorization and corresponding TT rank, which can well express the low-rankness and mode correlations of higher-order tensors, have attracted much attention in recent years. However, TT factorization based methods are generally not sufficient to characterize low-rankness along each mode of third-order tensor. Inspired by this, we generalize the tensor train factorization to the mode-k tensor train factorization and introduce a corresponding multi-mode tensor train (MTT) rank. Then, we proposed a novel low-MTT-rank tensor completion model via multi-mode TT factorization and s…
The geometry of the set of restrictions of rank-one tensors to some of their coordinates is studied. This gives insight into the problem of rank-one completion of partial tensors. Particular emphasis is put on the semialgebraic nature of the problem, which arises for real tensors with constraints on the parameters. The algebraic boundary of the completable region is described for tensors parametrized by probability distributions and where the number of observed entries equals the number of parameters. If the observations are on the diagonal of a tensor of format $d\times\dots\times d$, the com…
