The paper is devoted to the efficient computation of high-order cubature formulas for volume potentials obtained within the framework of approximate approximations. We combine this approach with modern methods of structured tensor product approximations. Instead of performing high-dimensional discrete convolutions the cubature of the potentials can be reduced to a certain number of one-dimensional convolutions leading to a considerable reduction of computing resources. We propose one-dimensional integral representions of high-order cubature formulas for n-dimensional harmonic and Yukawa potentials, which allow low rank tensor product approximations.
There exist linear relations among tensor entries of low rank tensors. These linear relations can be expressed by multi-linear polynomials, which are called generating polynomials. We use generating polynomials to compute tensor rank decompositions and low rank tensor approximations. We prove that this gives a quasi-optimal low rank tensor approximation if the given tensor is sufficiently close to a low rank one.
The problem of partitioning a large and sparse tensor is considered, where the tensor consists of a sequence of adjacency matrices. Theory is developed that is a generalization of spectral graph partitioning. A best rank-$(2,2,λ)$ approximation is computed for $λ=1,2,3$, and the partitioning is computed from the orthogonal matrices and the core tensor of the approximation. It is shown that if the tensor has a certain reducibility structure, then the solution of the best approximation problem exhibits the reducibility structure of the tensor. Further, if the tensor is close to being reducible, …
In this article, we analyze tensor approximation schemes for continuous functions. We assume that the function to be approximated lies in an isotropic Sobolev space and discuss the cost when approximating this function in the continuous analogue of the Tucker tensor format or of the tensor train format. We especially show that the cost of both approximations are dimension-robust when the Sobolev space under consideration provides appropriate weights.
In this paper, we define a semi-tensor product for third-order tensors. Based on this definition, we present a new type of tensor decomposition strategy and give the specific algorithm. This decomposition strategy actually generalizes the tensor SVD based on semi-tensor product. Due to the characteristic of semi-tensor product for compressing the data scale, we can therefore achieve data compression in this way. Numerical comparisons are given to show the advantages of this decomposition.
We present an orthogonal matrix outer product decomposition for the fourth-order conjugate partial-symmetric (CPS) tensor and show that the greedy successive rank-one approximation (SROA) algorithm can recover this decomposition exactly. Based on this matrix decomposition, the CP rank of CPS tensor can be bounded by the matrix rank, which can be applied to low rank tensor completion. Additionally, we give the rank-one equivalence property for the CPS tensor based on the SVD of matrix, which can be applied on the rank-one approximation for CPS tensors.
This paper establishes some perturbation analysis for the tensor inverse, the tensor Moore-Penrose inverse and the tensor system based on the t-product. In the settings of structured perturbations, we generalize the Sherman-Morrison-Woodbury (SMW) formula to the t-product tensor scenarios. The SMW formula can be used to perform sensitivity analysis for a multilinear system of equations.
Low-rank approximation of tensors has been widely used in high-dimensional data analysis. It usually involves singular value decomposition (SVD) of large-scale matrices with high computational complexity. Sketching is an effective data compression and dimensionality reduction technique applied to the low-rank approximation of large matrices. This paper presents two practical randomized algorithms for low-rank Tucker approximation of large tensors based on sketching and power scheme, with a rigorous error-bound analysis. Numerical experiments on synthetic and real-world tensor data demonstrate …
Information is extracted from large and sparse data sets organized as 3-mode tensors. Two methods are described, based on best rank-(2,2,2) and rank-(2,2,1) approximation of the tensor. The first method can be considered as a generalization of spectral graph partitioning to tensors, and it gives a reordering of the tensor that clusters the information. The second method gives an expansion of the tensor in sparse rank-(2,2,1) terms, where the terms correspond to graphs. The low-rank approximations are computed using an efficient Krylov-Schur type algorithm that avoids filling in the sparse data…
The paper is concerned with methods for computing the best low multilinear rank approximation of large and sparse tensors. Krylov-type methods have been used for this problem; here block versions are introduced. For the computation of partial eigenvalue and singular value decompositions of matrices the Krylov-Schur (restarted Arnoldi) method is used. We describe a generalization of this method to tensors, for computing the best low multilinear rank approximation of large and sparse tensors. In analogy to the matrix case, the large tensor is only accessed in multiplications between the tensor a…
In the present paper, we introduce new tensor Krylov subspace methods for solving linear tensor equations. The proposed methods use the well known T-product for tensors and tensor subspaces related to tube fibers. We introduce some new tensor products and the related algebraic properties. These new products will enable us to develop third-order the tensor tubal GMRES and the tensor tubal Golub Kahan methods. We give some properties related to these methods and proopse some numerical experiments.
In this paper, we introduce a method for multivariate function approximation using function evaluations, Chebyshev polynomials, and tensor-based compression techniques via the Tucker format. We develop novel randomized techniques to accomplish the tensor compression, provide a detailed analysis of the computational costs, provide insight into the error of the resulting approximations, and discuss the benefits of the proposed approaches. We also apply the tensor-based function approximation to develop low-rank matrix approximations to kernel matrices that describe pairwise interactions between …
Tensor trains are a versatile tool to compress and work with high-dimensional data and functions. In this work we introduce the Streaming Tensor Train Approximation (STTA), a new class of algorithms for approximating a given tensor $\mathcal T$ in the tensor train format. STTA accesses $\mathcal T$ exclusively via two-sided random sketches of the original data, making it streamable and easy to implement in parallel -- unlike existing deterministic and randomized tensor train approximations. This property also allows STTA to conveniently leverage structure in $\mathcal T$, such as sparsity and …
This paper studies the issues about tensors. Three typical kinds of tensor decomposition are mentioned. Among these decompositions, the t-SVD is proposed in this decade. Different definitions of rank derive from tensor decompositions. Based on the research about higher order tensor t-product and tensor products with invertible transform, this paper introduces a product performing higher order tensor products with invertible transform, which is the most generalized case so far. Also, a few properties are proven. Because the optimization model of low-rank recovery often uses the nuclear norm, th…
In the present work, a method is proposed in order to compute a Canonical Polyadic (CP) approximation of a given tensor. It is based on a greedy method and an adaptation of the TT-SVD method. The proposed approach can be straightforwardly extended to compute rank-$k$ updates in a stable way. Some numerical experiments are proposed, in which the proposed method is compared to ALS and ASVD methods and performs particularly well for high-order tensors.
In this paper, we focus on developing randomized algorithms for the computation of low multilinear rank approximations of tensors based on the random projection and the singular value decomposition. Following the theory of the singular values of sub-Gaussian matrices, we make a probabilistic analysis for the error bounds for the randomized algorithm. We demonstrate the effectiveness of proposed algorithms via several numerical examples.
Approximating higher-order tensors by the Tucker format has been applied in many fields such as psychometrics, chemometrics, signal processing, pattern classification, and so on. In this paper, we propose some new Tucker-like approximations based on the modal semi-tensor product (STP), especially, a new singular value decomposition (SVD) and a new higher-order SVD (HOSVD) are derived. Algorithms for computing new decompositions are provided. We also give some numerical examples to illustrate our theoretical results.
In this note, we present perturbation analysis for the T-product based tensor singular values defined by Lu et al. First, the Cauchy's interlacing-type theorem for tensor singular values is given. Then, the inequalities about the difference between the singular values of two matrices proposed by Mirsky are extended to tensor cases. Finally, we introduce some useful inequalities for the singular values of tensor products and sums.
Best Rank-One Tensor Approximation and Parallel Update Algorithm for CPD
Lab for Advanced Brain Signal Processing, Brain Science Institute, RIKEN（RIKEN脑科学研究所高级脑信号处理实验室）；Systems Research Institute PAS（波兰科学院系统研究所）；Skolkovo Institute of Science and Technology (Skoltech)（斯科尔科沃科学技术学院）；Institute of Information Theory and Automation（信息与自动化研究所）
We study the problem of finding orthogonal low-rank approximations of symmetric tensors. In the case of matrices, the approximation is a truncated singular value decomposition which is then symmetric. Moreover, for rank-one approximations of tensors of any dimension, a classical result proven by Banach in 1938 shows that the optimal approximation can always be chosen to be symmetric. In contrast to these results, this article shows that the corresponding statement is no longer true for orthogonal approximations of higher rank. Specifically, for any of the four common notions of tensor orthogon…
In this work, we investigate the tensor inequalities in the tensor t-product formalism. The inequalities involving tensor power are proved to hold similarly as standard matrix scenarios. We then focus on the tensor norm inequalities. The well-known arithmetic-geometric mean inequality, H{\" o}lder inequality, and Minkowski inequality are generalized to tensors. Furthermore, we obtain some t-eigenvalue inequalities.
We present a new method based on functional tensor decomposition and dynamic tensor approximation to compute the solution of a high-dimensional time-dependent nonlinear partial differential equation (PDE). The idea of dynamic approximation is to project the time derivative of the PDE solution onto the tangent space of a low-rank functional tensor manifold at each time. Such a projection can be computed by minimizing a convex energy functional over the tangent space. This minimization problem yields the unique optimal velocity vector that allows us to integrate the PDE forward in time on a tens…
A gradient-enhanced functional tensor train cross approximation method for the resolution of the Hamilton-Jacobi-Bellman (HJB) equations associated to optimal feedback control of nonlinear dynamics is presented. The procedure uses samples of both the solution of the HJB equation and its gradient to obtain a tensor train approximation of the value function. The collection of the data for the algorithm is based on two possible techniques: Pontryagin Maximum Principle and State Dependent Riccati Equations. Several numerical tests are presented in low and high dimension showing the effectiveness o…
Sparse tensor best rank-1 approximation (BR1Approx), which is a sparsity generalization of the dense tensor BR1Approx, and is a higher-order extension of the sparse matrix BR1Approx, is one of the most important problems in sparse tensor decomposition and related problems arising from statistics and machine learning. By exploiting the multilinearity as well as the sparsity structure of the problem, four approximation algorithms are proposed, which are easily implemented, of low computational complexity, and can serve as initial procedures for iterative algorithms. In addition, theoretically gu…
The University of Hong Kong（香港大学）；Weifang University（潍坊学院）；Southwestern University of Finance and Economics（西南财经大学）
The main aim of this paper is to develop a new algorithm for computing nonnegative low rank tensor approximation for nonnegative tensors that arise in many multi-dimensional imaging applications. Nonnegativity is one of the important property as each pixel value refers to nonzero light intensity in image data acquisition. Our approach is different from classical nonnegative tensor factorization (NTF) which requires each factorized matrix and/or tensor to be nonnegative. In this paper, we determine a nonnegative low Tucker rank tensor to approximate a given nonnegative tensor. We propose an alt…
University of California, Los Angeles（加州大学洛杉矶分校）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）
Low rank tensor approximation is a fundamental tool in modern machine learning and data science. In this paper, we study the characterization, perturbation analysis, and an efficient sampling strategy for two primary tensor CUR approximations, namely Chidori and Fiber CUR. We characterize exact tensor CUR decompositions for low multilinear rank tensors. We also present theoretical error bounds of the tensor CUR approximations when (adversarial or Gaussian) noise appears. Moreover, we show that low cost uniform sampling is sufficient for tensor CUR approximations if the tensor has an incoherent…
Boston College（波士顿学院）；Brandeis University（布兰迪斯大学）；Boston University（波士顿大学）；J.P. Morgan（摩根大通）
We analyze the problem of high-order polynomial approximation from a many-body physics perspective, and demonstrate the descriptive power of entanglement entropy in capturing model capacity and task complexity. Instantiated with a high-order nonlinear dynamics modeling problem, tensor-network models are investigated and exhibit promising modeling advantages. This novel perspective establish a connection between quantum information and functional approximation, which worth further exploration in future research.
