46 pages; some major adjustments and additions in Section 2 (especially on castling transforms) and Section 6; further minor revisions in other sections
These lecture notes are intended as an introduction to several notions of tensor rank and their connections to the asymptotic complexity of matrix multiplication. The latter is studied with the exponent of matrix multiplication, which will be expressed in terms of tensor (border) rank, (border) symmetric rank and the asymptotic rank of certain tensors. We introduce the multilinear rank of a tensor as well, deal with the concept of tensor equivalence and study prehomogeneous vector spaces with the castling transform. Moreover, we treat Apolarity Theory and use it to determine the symmetric rank (Waring rank) of some symmetric tensors.
For odd n, I write down tensors in C^n\otimes C^n\otimes C^n of border rank 2n-1, showing the non-triviality of the Young-flattening equations of Landsberg-Ottaviani. I also study the border rank of the tensors of Alexeev et. al., showing the tensors their tensors T_{2^k}, despite having rank equal to 2^{k+1}-1, have border rank equal to 2^k, the minimum of any concise tensor. I also study the equations of Griesser.
Whereas matrix rank is additive under direct sum, in 1981 Schönhage showed that one of its generalizations to the tensor setting, tensor border rank, can be strictly subadditive for tensors of order three. Whether border rank is additive for higher order tensors has remained open. In this work, we settle this problem by providing analogs of Schönhage's construction for tensors of order four and higher. Schönhage's work was motivated by the study of the computational complexity of matrix multiplication; we discuss implications of our results for the asymptotic rank of higher order generalizatio…
We prove that the border rank of the Kronecker square of the little Coppersmith-Winograd tensor $T_{cw,q}$ is the square of its border rank for $q > 2$ and that the border rank of its Kronecker cube is the cube of its border rank for $q > 4$. This answers questions raised implicitly in [Coppersmith-Winograd, 1990] and explicitly in [Bläser, 2013] and rules out the possibility of proving new upper bounds on the exponent of matrix multiplication using the square or cube of a little Coppersmith-Winograd tensor in this range. In the positive direction, we enlarge the list of explicit tensors poten…
There are close relations between tripartite tensors with bounded geometric ranks and linear determinantal varieties with bounded codimensions. We study linear determinantal varieties with bounded codimensions, and prove upper bounds of the dimensions of the ambient spaces. Using those results, we classify tensors with geometric rank 3, find upper bounds of multilinear ranks of primitive tensors with geometric rank 4, and prove the existence of such upper bounds in general. We extend results of tripartite tensors to n-part tensors, showing the equivalence between geometric rank 1 and partition…
We establish basic information about border rank algorithms for the matrix multiplication tensor and other tensors with symmetry. We prove that border rank algorithms for tensors with symmetry (such as matrix multiplication and the determinant polynomial) come in families that include representatives with normal forms. These normal forms will be useful both to develop new efficient algorithms and to prove lower complexity bounds. We derive a border rank version of the substitution method used in proving lower bounds for tensor rank. We use this border-substitution method and a normal form to i…
We make a geometric study of the Geometric Rank of tensors recently introduced by Kopparty et al. Results include classification of tensors with degenerate geometric rank in $C^3\otimes C^3\otimes C^3$, classification of tensors with geometric rank two, and showing that upper bounds on geometric rank imply lower bounds on tensor rank.
It has been shown that a best rank-R approximation of an order-k tensor may not exist when R>1 and k>2. This poses a serious problem to data analysts using tensor decompositions. It has been observed numerically that, generally, this issue cannot be solved by consecutively computing and subtracting best rank-1 approximations. The reason for this is that subtracting a best rank-1 approximation generally does not decrease tensor rank. In this paper, we provide a mathematical treatment of this property for real-valued 2x2x2 tensors, with symmetric tensors as a special case. Regardless of the symm…
Fix integers $m \ge 2$, $s\ge 5$ and $d\ge 2s+2$. Here we describe the possible symmetric tensor ranks $\le 2d+s-7$ of all symmetric tensors (or homogeneous degree $d$ polynomials) in $m+1$ variables with border rank $s$.
We lower bound the rank of a tensor by a linear combination of the ranks of three of its unfoldings, using Sylvester's rank inequality. In a similar way, we lower bound the symmetric rank by a linear combination of the symmetric ranks of three unfoldings. Lower bounds on the rank and symmetric rank of tensors are important for finding counterexamples to Comon's conjecture. A real counterexample to Comon's conjecture is a tensor whose real rank and real symmetric rank differ. Previously, only one real counterexample was known. We divide the construction into three steps. The first step involves…
Hankel tensors are generalizations of Hankel matrices. This article studies the relations among various ranks of Hankel tensors. We give an algorithm that can compute the Vandermonde ranks and decompositions for all Hankel tensors. For a generic $n$-dimensional Hankel tensor of even order or order three, we prove that the the cp rank, symmetric rank, border rank, symmetric border rank, and Vandermonde rank all coincide with each other. In particular, this implies that the Comon's conjecture is true for generic Hankel tensors when the order is even or three. Some open questions are also posed.
We prove (without exceptions) the existence of irredundant tensor decompositions with the number of addenda equal to rank $+1$. We also discuss the existence of decompositions with more than the tensor rank terms, which are concise, while the original tensor is not concise.
Shitov recently gave a counterexample to Comon's conjecture that the symmetric tensor rank and tensor rank of a symmetric tensor are the same. In this paper we show that an analog of Comon's conjecture for the G-stable rank introduced by Derksen is true: the symmetric G-stable rank and G-stable rank of a symmetric tensor are the same. We also show that the log-canonical threshold of a singularity is bounded by the G-stable rank of the defining ideal.
We introduce various notions of rank for a symmetric tensor, namely: rank, border rank, catalecticant rank, generalized rank, scheme length, border scheme length, extension rank and smoothable rank. We analyze the stratification induced by these ranks. The mutual relations between these stratifications, allow us to describe the hierarchy among all the ranks. We show that strict inequalities are possible between rank, border rank, extension rank and catalecticant rank. Moreover we show that scheme length, generalized rank and extension rank coincide.
Rank-2 and rank-3 tensors are almost all identifiable with only few exceptions. We classify them all together with the dimensions and the structures of all the sets evincing the rank.
We introduce subspace rank as a tool for studying ranks of tensors and X-rank more generally. We derive a new upper bound for the rank of a tensor and determine the ranks of partially symmetric tensors in C^2 \otimes C^b \otimes C^b. We review the literature from a geometric perspective.
We study tensors in $\C^{m\times n\times l}$ whose border rank is $l$. We characterize the tensors in $\C^{3\times 3\times 4}$ and in $\C^{4\times 4\times 4}$ of border rank 4 at most.
Motivated by questions arising in signal processing, computational complexity, and other areas, we study the ranks and border ranks of symmetric tensors using geometric methods. We provide improved lower bounds for the rank of a symmetric tensor (i.e., a homogeneous polynomial) obtained by considering the singularities of the hypersurface defined by the polynomial. We obtain normal forms for polynomials of border rank up to five, and compute or bound the ranks of several classes of polynomials, including monomials, the determinant, and the permanent.
We describe the stratification by tensor rank of the points belonging to the tangent developable of any Segre variety. We give algorithms to compute the rank and a decomposition of a tensor belonging to the secant variety of lines of any Segre variety. We prove Comon's conjecture on the rank of symmetric tensors for those tensors belonging to tangential varieties to Veronese varieties.
Inspired by recent work of Kopparty-Moshkovitz-Zuiddam and motivated by problems in combinatorics and hypergraphs, we introduce the notion of the symmetric geometric rank of a symmetric tensor. This quantity is equal to the codimension of the singular locus of the hypersurface associated to the tensor. We first derive fundamental properties of the symmetric geometric rank. Then, we study the space of symmetric tensors of prescribed symmetric geometric rank, which are spaces of homogeneous polynomials whose corresponding hypersurfaces have a singular locus of bounded codimension.
In this paper, we give a survey of the known results concerning the tensor rank of the multiplication in finite fields and we establish new asymptotical and not asymptotical upper bounds about it.
We consider the problem of determining the symmetric tensor rank for symmetric tensors with an algebraic geometry approach. We give algorithms for computing the symmetric rank for $2\times ... \times 2$ tensors and for tensors of small border rank. From a geometric point of view, we describe the symmetric rank strata for some secant varieties of Veronese varieties.
It is shown that the relative distance in Frobenius norm of a real symmetric order-$d$ tensor of rank two to its best rank-one approximation is upper bounded by $\sqrt{1-(1-1/d)^{d-1}}$. This is achieved by determining the minimal possible ratio between spectral and Frobenius norm for symmetric tensors of border rank two, which equals $\left(1-{1}/{d}\right)^{(d-1)/{2}}$. These bounds are also verified for arbitrary real rank-two tensors by reducing to the symmetric case.
The article is concerned with the problem of the additivity of the tensor rank. That is for two independent tensors we study when the rank of their direct sum is equal to the sum of their individual ranks. The statement saying that additivity always holds was previously known as Strassen's conjecture (1969) until Shitov proposed counterexamples (2019). They are not explicit and only known to exist asymptotically for very large tensor spaces. In this article, we show that for some small three-way tensors the additivity holds. For instance, we give a proof that another conjecture stated by Stras…
We introduce an inductive method for the study of the uniqueness of decompositions of tensors, by means of tensors of rank 1. The method is based on the geometric notion of weak defectivity. For three-dimensional tensors of type (a, b, c), a\le b\le c, our method proves that the decomposition is unique (i.e. k-identifiability holds) for general tensors of rank k, as soon as k\le (a+1)(b+1)/16. This improves considerably the known range for identifiability. The method applies also to tensor of higher dimension. For tensors of small size, we give a complete list of situations where identifiabili…
We discover a geometric property of the space of tensors of fixed multilinear (Tucker) rank. Namely, it is shown that real tensors of fixed multilinear rank form a minimal submanifold of the Euclidean space of tensors endowed with the Frobenius inner product. We also establish the absence of local extrema for linear functionals restricted to the submanifold of rank-one tensors, finding application in statistics.
We address the problem of the additivity of the tensor rank. That is for two independent tensors we study if the rank of their direct sum is equal to the sum of their individual ranks. A positive answer to this problem was previously known as Strassen's conjecture until recent counterexamples were proposed by Shitov. The latter are not very explicit, and they are only known to exist asymptotically for very large tensor spaces. In this article we prove that for some small three-way tensors the additivity holds. For instance, if the rank of one of the tensors is at most 6, then the additivity ho…
It has recently been shown that the tensor rank can be strictly submultiplicative under the tensor product, where the tensor product of two tensors is a tensor whose order is the sum of the orders of the two factors. The necessary upper bounds were obtained with help of border rank. It was left open whether border rank itself can be strictly submultiplicative. We answer this question in the affirmative. In order to do so, we construct lines in projective space along which the border rank drops multiple times and use this result in conjunction with a previous construction for a tensor rank drop…
Given a tensor $f$ in a Euclidean tensor space, we are interested in the critical points of the distance function from $f$ to the set of tensors of rank at most $k$, which we call the critical rank-at-most-$k$ tensors for $f$. When $f$ is a matrix, the critical rank-one matrices for $f$ correspond to the singular pairs of $f$. The critical rank-one tensors for $f$ lie in a linear subspace $H_f$, the critical space of $f$. Our main result is that, for any $k$, the critical rank-at-most-$k$ tensors for a sufficiently general $f$ also lie in the critical space $H_f$. This is the part of Eckart-Yo…
We give a sufficient criterion for a lower bound of the cactus rank of a tensor. Then we refine that criterion in order to be able to give an explicit sufficient condition for a non-redundant decomposition of a tensor to be minimal and unique.
Motivated by the many potential applications of low-rank multi-way tensor approximations, we set out to count the rank-one tensors that are critical points of the distance function to a general tensor v. As this count depends on v, we average over v drawn from a Gaussian distribution, and find formulas that relates this average to problems in random matrix theory.
A nonnegative tensor has nonnegative rank at most 2 if and only if it is supermodular and has flattening rank at most 2. We prove this result, then explore the semialgebraic geometry of the general Markov model on phylogenetic trees with binary states, and comment on possible extensions to tensors of higher rank.
Laboratoire I3S, CNRS and the University of Nice, Sophia-Antipolis（I3S实验室，法国国家科学研究中心和尼斯大学）；Department of Computer Science and the Institute for Computational and Mathematical Engineering, Stanford University（计算机科学系和计算与数学工程研究所，斯坦福大学）；Projet GALAAD, INRIA（GALAAD项目，法国国家信息与自动化研究所）
We introduce the $G$-stable rank of a higher order tensors over perfect fields. The $G$-stable rank is related to the Hilbert-Mumford criterion for stability in Geometric Invariant Theory. We will relate the $G$-stable rank to the tensor rank and slice rank. For numerical applications, we express the $G$-stable rank as a solution to an optimization problem. Over the field ${\mathbb F}_3$ we discuss an application to the Cap Set Problem.
Given tensors $T$ and $T'$ of order $k$ and $k'$ respectively, the tensor product $T \otimes T'$ is a tensor of order $k+k'$. It was recently shown that the tensor rank can be strictly submultiplicative under this operation ([Christandl-Jensen-Zuiddam]). We study this phenomenon for symmetric tensors where additional techniques from algebraic geometry become available. The tensor product of symmetric tensors results in a partially symmetric tensor and our results amount to bounds on the partially symmetric rank. Following motivations from algebraic complexity theory and quantum information the…
We use an algebraic approach to construct minimal decompositions of symmetric tensors with low rank. This is done by using Apolarity Theory and by studying minimal sets of reduced points apolar to a given symmetric tensor, namely, whose ideal is contained in the apolar ideal associated to the tensor. In particular, we focus on the structure of the Hilbert function of these ideals of points. We give a procedure which produces a minimal set of points apolar to any symmetric tensor of rank at most 5. This procedure is also implemented in the algebra software Macaulay2.
GIPSA-Lab CNRS（GIPSA实验室 法国国家科学研究中心）；University of Chicago（芝加哥大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）
The canonical polyadic decomposition (CPD) of a low rank tensor plays a major role in data analysis and signal processing by allowing for unique recovery of underlying factors. However, it is well known that the low rank CPD approximation problem is ill-posed. That is, a tensor may fail to have a best rank $R$ CPD approximation when $R>1$. This article gives deterministic bounds for the existence of best low rank tensor approximations over $\mathbb{K}=\mathbb{R}$ or $\mathbb{K}=\mathbb{C}$. More precisely, given a tensor $\mathcal{T} \in \mathbb{K}^{I \times I \times I}$ of rank $R \leq I$, we…
Let $N_1, \ldots, N_d$ be positive integers with $N_1\leq\cdots\leq N_d$. Set $N=N_1\cdots N_{d-1}$. We show in this paper that an integer $r$ is a typical nonnegative rank of nonnegative tensors of format $N_1\times\cdots\times N_d$ if and only if $r\leq N$ and $r$ is greater than or equals to the generic rank of tensors over $\mathbb{C}$ of format $N_1\times\cdots\times N_d$. We also show that the maximal nonnegative rank of nonnegative tensors of format $N_1\times\cdots\times N_d$ is $N$.
First-order methods on bounded-rank tensors converging to stationary points
State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Chinese Academy of Sciences（中国科学院大学）
Though algebraic geometry over $\mathbb C$ is often used to describe the closure of the tensors of a given size and complex rank, this variety includes tensors of both smaller and larger rank. Here we focus on the $n\times n\times n$ tensors of rank $n$ over $\mathbb C$, which has as a dense subset the orbit of a single tensor under a natural group action. We construct polynomial invariants under this group action whose non-vanishing distinguishes this orbit from points only in its closure. Together with an explicit subset of the defining polynomials of the variety, this gives a semialgebraic …
