We describe a simple, black-box compression format for tensors with a multiscale structure. By representing the tensor as a sum of compressed tensors defined on increasingly coarse grids, we capture low-rank structures on each grid-scale, and we show how this leads to an increase in compression for a fixed accuracy. We devise an alternating algorithm to represent a given tensor in the multiresolution format and prove local convergence guarantees. In two dimensions, we provide examples that show that this approach can beat the Eckart-Young theorem, and for dimensions higher than two, we achieve higher compression than the tensor-train format on six real-world datasets. We also provide results on the closedness and stability of the tensor format and discuss how to perform common linear algebra operations on the level of the compressed tensors.
There exist linear relations among tensor entries of low rank tensors. These linear relations can be expressed by multi-linear polynomials, which are called generating polynomials. We use generating polynomials to compute tensor rank decompositions and low rank tensor approximations. We prove that this gives a quasi-optimal low rank tensor approximation if the given tensor is sufficiently close to a low rank one.
We present an orthogonal matrix outer product decomposition for the fourth-order conjugate partial-symmetric (CPS) tensor and show that the greedy successive rank-one approximation (SROA) algorithm can recover this decomposition exactly. Based on this matrix decomposition, the CP rank of CPS tensor can be bounded by the matrix rank, which can be applied to low rank tensor completion. Additionally, we give the rank-one equivalence property for the CPS tensor based on the SVD of matrix, which can be applied on the rank-one approximation for CPS tensors.
In this paper, we introduce a new tensor decomposition for third order tensors, which decomposes a third order tensor to three third order low rank tensors in a balanced way. We call such a decomposition the triple decomposition, and the corresponding rank the triple rank. For a third order tensor, its CP decomposition can be regarded as a special case of its triple decomposition. The triple rank of a third order tensor is not greater than the middle value of the Tucker rank, and is strictly less than the middle value of the Tucker rank for an essential class of examples. These indicate that p…
This paper considers the completion problem for a tensor (also referred to as a multidimensional array) from limited sampling. Our greedy method is based on extending the low-rank approximation pursuit (LRAP) method for matrix completions to tensor completions. The method performs a tensor factorization using the tensor singular value decomposition (t-SVD) which extends the standard matrix SVD to tensors. The t-SVD leads to a notion of rank, called tubal-rank here. We want to recreate the data in tensors from low resolution samples as best we can here. To complete a low resolution tensor succe…
Information is extracted from large and sparse data sets organized as 3-mode tensors. Two methods are described, based on best rank-(2,2,2) and rank-(2,2,1) approximation of the tensor. The first method can be considered as a generalization of spectral graph partitioning to tensors, and it gives a reordering of the tensor that clusters the information. The second method gives an expansion of the tensor in sparse rank-(2,2,1) terms, where the terms correspond to graphs. The low-rank approximations are computed using an efficient Krylov-Schur type algorithm that avoids filling in the sparse data…
The orthogonal decomposition factorizes a tensor into a sum of an orthogonal list of rankone tensors. We present several properties of orthogonal rank. We find that a subtensor may have a larger orthogonal rank than the whole tensor and prove the lower semicontinuity of orthogonal rank. The lower semicontinuity guarantees the existence of low orthogonal rank approximation. To fit the orthogonal decomposition, we propose an algorithm based on the augmented Lagrangian method and guarantee the orthogonality by a novel orthogonalization procedure. Numerical experiments show that the proposed metho…
A matrix always has a full rank submatrix such that the rank of this matrix is equal to the rank of that submatrix. This property is one of the corner stones of the matrix rank theory. We call this property the max-full-rank-submatrix property. Tensor ranks play a crucial role in low rank tensor approximation, tensor completion and tensor recovery. However, their theory is still not matured yet. Can we set an axiom system for tensor ranks? Can we extend the max-full-rank-submatrix property to tensors? We explore these in this paper. We first propose some axioms for tensor rank functions. Then …
We consider the low-rank tensor train completion problem when additional side information is available in the form of subspaces that contain the mode-$k$ fiber spans. We propose an algorithm based on Riemannian optimization to solve the problem. Numerical experiments show that the proposed algorithm requires far fewer known entries to recover the tensor compared to standard tensor train completion methods.
Uncertainty quantification based on stochastic spectral methods suffers from the curse of dimensionality. This issue was mitigated recently by low-rank tensor methods. However, there exist two fundamental challenges in low-rank tensor-based uncertainty quantification: how to automatically determine the tensor rank and how to pick the simulation samples. This paper proposes a novel tensor regression method to address these two challenges. Our method uses an $\ell_{q}/ \ell_{2}$-norm regularization to determine the tensor rank and an estimated Voronoi diagram to pick informative samples for simu…
Low-rank approximation of tensors has been widely used in high-dimensional data analysis. It usually involves singular value decomposition (SVD) of large-scale matrices with high computational complexity. Sketching is an effective data compression and dimensionality reduction technique applied to the low-rank approximation of large matrices. This paper presents two practical randomized algorithms for low-rank Tucker approximation of large tensors based on sketching and power scheme, with a rigorous error-bound analysis. Numerical experiments on synthetic and real-world tensor data demonstrate …
In 2011, Kilmer and Martin proposed tensor singular value decomposition (T-SVD) for third order tensors. Since then, T-SVD has applications in low rank tensor approximation, tensor recovery, multi-view clustering, multi-view feature extraction, tensor sketching, etc. By going through the Discrete Fourier Transform (DFT), matrix SVD and inverse DFT, a third order tensor is mapped to an f-diagonal third order tensor. We call this a Kilmer-Martin mapping. We show that the Kilmer-Martin mapping of a third order tensor is invariant if that third order tensor is taking T-product with some orthogonal…
In this paper, we consider the rank-one tensor completion problem. We address the question of existence and uniqueness of the rank-one solution. In particular we show that the global uniqueness over the field of real numbers can be verified in a polynomial time. We give examples showing that there is an essential difference between the question of global uniqueness over the fields of real and complex numbers. Finally we briefly discuss the rank-one approximation problem for noisy observations.
Tensor numerical methods, based on the rank-structured tensor representation of $d$-variate functions and operators, are designed to provide $O(dn)$ complexity of numerical calculations on $n^{\otimes d }$ grids contrary to $O(n^d)$ scaling by conventional grid-based methods. However, multiple tensor operations may lead to enormous increase in the tensor ranks (curse of ranks) of the target data, making calculation intractable. Therefore one of the most important steps in tensor calculations is the robust and efficient rank reduction procedure which should be performed many times in the course…
Best Rank-One Tensor Approximation and Parallel Update Algorithm for CPD
Lab for Advanced Brain Signal Processing, Brain Science Institute, RIKEN（RIKEN脑科学研究所高级脑信号处理实验室）；Systems Research Institute PAS（波兰科学院系统研究所）；Skolkovo Institute of Science and Technology (Skoltech)（斯科尔科沃科学技术学院）；Institute of Information Theory and Automation（信息与自动化研究所）
Laboratoire I3S, CNRS and the University of Nice, Sophia-Antipolis（I3S实验室，法国国家科学研究中心和尼斯大学）；Department of Computer Science and the Institute for Computational and Mathematical Engineering, Stanford University（计算机科学系和计算与数学工程研究所，斯坦福大学）；Projet GALAAD, INRIA（GALAAD项目，法国国家信息与自动化研究所）
Tensor Deflation for CANDECOMP/PARAFAC. Part 3: Rank Splitting
BSI-TOYOTA Collaboration Center, RIKEN（理化学研究所BSI-丰田合作中心）；System Research Institute, Warsaw（华沙系统研究所）；Institute of Information Theory and Automation, Prague（布拉格信息与自动化研究所）
University of Electronic Science and Technology of China（电子科技大学）
Tensor completion is a challenging problem with various applications. Many related models based on the low-rank prior of the tensor have been proposed. However, the low-rank prior may not be enough to recover the original tensor from the observed incomplete tensor. In this paper, we prose a tensor completion method by exploiting both the low-rank and sparse prior of tensor. Specifically, the tensor completion task can be formulated as a low-rank minimization problem with a sparse regularizer. The low-rank property is depicted by the tensor truncated nuclear norm based on tensor singular value …
The canonical polyadic decomposition (CPD) of a low rank tensor plays a major role in data analysis and signal processing by allowing for unique recovery of underlying factors. However, it is well known that the low rank CPD approximation problem is ill-posed. That is, a tensor may fail to have a best rank $R$ CPD approximation when $R>1$. This article gives deterministic bounds for the existence of best low rank tensor approximations over $\mathbb{K}=\mathbb{R}$ or $\mathbb{K}=\mathbb{C}$. More precisely, given a tensor $\mathcal{T} \in \mathbb{K}^{I \times I \times I}$ of rank $R \leq I$, we…
Multiway data often naturally occurs in a tensorial format which can be approximately represented by a low-rank tensor decomposition. This is useful because complexity can be significantly reduced and the treatment of large-scale data sets can be facilitated. In this paper, we find a low-rank representation for a given tensor by solving a Bayesian inference problem. This is achieved by dividing the overall inference problem into sub-problems where we sequentially infer the posterior distribution of one tensor decomposition component at a time. This leads to a probabilistic interpretation of th…
A tensor is a multi-way array that can represent, in addition to a data set, the expression of a joint law or a multivariate function. As such it contains the description of the interactions between the variables corresponding to each of the entries. The rank of a tensor extends to arrays with more than two entries the notion of rank of a matrix, bearing in mind that there are several approaches to build such an extension. When the rank is one, the variables are separated, and when it is low, the variables are weakly coupled. Many calculations are simpler on tensors of low rank. Furthermore, a…
GIPSA-Lab CNRS（GIPSA实验室 法国国家科学研究中心）；University of Chicago（芝加哥大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）
Boston College（波士顿学院）；J.P. Morgan（摩根大通）；Brandeis University（布兰迪斯大学）；Boston University（波士顿大学）
This study clarifies the proper criteria to assess the modeling capacity of a general tensor model. The work analyze the problem based on the study of tensor ranks, which is not a well-defined quantity for higher order tensors. To process, the author introduces the separability issue to discuss the Cannikin's law of tensor modeling. Interestingly, a connection between entanglement studied in information theory and tensor analysis is established, shedding new light on the theoretical understanding for modeling capacity problems.
First-order methods on bounded-rank tensors converging to stationary points
State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Chinese Academy of Sciences（中国科学院大学）
