Tensor rank and the ill-posedness of the best low-rank approximation problem
Vin de Silva, Lek-Heng Lim
备注:
To appear in the SIAM Journal on Matrix Analysis and Applications. Includes another 7 pages of additional materials not in the earlier Tech Report version
There has been continued interest in seeking a theorem describing optimal low-rank approximations to tensors of order 3 or higher, that parallels the Eckart-Young theorem for matrices. In this paper, we argue that the naive approach to this problem is doomed to failure because, unlike matrices, tensors of order 3 or higher can fail to have best rank-r approximations. The phenomenon is much more widespread than one might suspect: examples of this failure can be constructed over a wide range of dimensions, orders and ranks, regardless of the choice of norm (or even Bregman divergence). Moreover, we show that in many instances these counterexamples have positive volume: they cannot be regarded as isolated phenomena. In one extreme case, we exhibit a tensor space in which no rank-3 tensor has an optimal rank-2 approximation. The notable exceptions to this misbehavior are rank-1 tensors and order-2 tensors. In a more positive spirit, we propose a natural way of overcoming the ill-posedness of the low-rank approximation problem, by using weak solutions when true solutions do not exist. In our work we emphasize the importance of closely studying concrete low-dimensional examples as a first step towards more general results. To this end, we present a detailed analysis of equivalence classes of 2-by-2-by-2 tensors, and we develop methods for extending results upwards to higher orders and dimensions. Finally, we link our work to existing studies of tensors from an algebraic geometric point of view. The rank of a tensor can in theory be given a semialgebraic description; i.e., can be determined by a system of polynomial inequalities. In particular we make extensive use of the 2-by-2-by-2 hyperdeterminant.
First-order methods on bounded-rank tensors converging to stationary points
State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院);University of Chinese Academy of Sciences(中国科学院大学)
Laboratoire I3S, CNRS and the University of Nice, Sophia-Antipolis(I3S实验室,法国国家科学研究中心和尼斯大学);Department of Computer Science and the Institute for Computational and Mathematical Engineering, Stanford University(计算机科学系和计算与数学工程研究所,斯坦福大学);Projet GALAAD, INRIA(GALAAD项目,法国国家信息与自动化研究所)
Tensor Deflation for CANDECOMP/PARAFAC. Part 3: Rank Splitting
BSI-TOYOTA Collaboration Center, RIKEN(理化学研究所BSI-丰田合作中心);System Research Institute, Warsaw(华沙系统研究所);Institute of Information Theory and Automation, Prague(布拉格信息与自动化研究所)
GIPSA-Lab CNRS(GIPSA实验室 法国国家科学研究中心);University of Chicago(芝加哥大学);Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院)
Reduced Higher Order SVD: ubiquitous rank-reduction method in tensor-based scientific computing
张量数值方法,基于秩结构的张量表示的$d$变量函数和运营商,旨在提供$O(dn)$复杂性的数值计算$n^{\otimes d }$网格相反的$O(n^d)$缩放传统的基于网格的方法。然而,多个张量操作可能导致目标数据的张量秩(秩的诅咒)的巨大增加,使得计算变得棘手。因此,在张量计算中最重要的步骤之一是鲁棒的和有效的降秩过程,在多维算子和函数演算中的各种张量变换的过程中,应该多次执行。在文献[33]中引入的基于降阶高阶奇异值分解(RHOSVD)的降秩方法在张量数值方法的发展中起了重要作用。在这里,我们简要地调查RHOSVD方法的要点,然后集中在一些新的理论和计算方面的RHOSVD证明,这种降秩技术构成的基本成分,在张量计算的现实生活中的问题。特别地,给出了RHOSVD的稳定性分析。我们介绍了张量的多重线性代数表示的范围分离(RS)张量格式。这使得RHOSVD降秩技术可以应用于具有许多奇异性的非规则函数数据,例如,生物分子建模中集体多粒子相互作用势的秩结构计算,以及复杂的复合径向函数。给出了RHOSVD在散乱数据建模中应用的新的理论和数值结果。RHOSVD被证明是一种有效的降秩技术,在许多应用中,从多粒子系统的数值处理到PDE约束控制问题的数值解。
Best Rank-One Tensor Approximation and Parallel Update Algorithm for CPD
Lab for Advanced Brain Signal Processing, Brain Science Institute, RIKEN(RIKEN脑科学研究所高级脑信号处理实验室);Systems Research Institute PAS(波兰科学院系统研究所);Skolkovo Institute of Science and Technology (Skoltech)(斯科尔科沃科学技术学院);Institute of Information Theory and Automation(信息与自动化研究所)