现代多核 CPU 上大稠密张量的低秩张量列 SVD（TT-SVD）性能
Performance of the low-rank tensor-train SVD (TT-SVD) for large dense tensors on modern multi-core CPUs
AI总结：
本文研究现代多核 CPU 上大稠密张量的低秩 TT-SVD 计算效率，提出基于无 Q 高瘦 QR 和融合高瘦矩阵乘法的算法，并通过 Roofline 模型和实验证明专用张量分解核心库可显著提升性能。
摘要：
多维张量存在多种分解为低维分量的分解方式，称为“张量网络”。我们考虑流行的“张量列”（TT）格式，并探讨：在当前多核 CPU 上，从完整张量计算低秩近似的效率如何？与稀疏和稠密线性代数相比，多重线性代数的核心库较为稀少，且通常优化程度不高。像 BLAS 和 LAPACK 这样的线性代数库原则上可以提供所需操作，但往往需要额外的数据移动来重新排列内存布局。此外，这些库通常针对计算受限的情况（例如方阵运算）进行优化，而低秩张量分解则导致内存带宽受限的操作。我们提出一种“张量列奇异值分解”（TT-SVD）算法，基于两个构建模块：一个“无 Q 的高瘦 QR”分解，以及一个融合的高瘦矩阵-矩阵乘法和重塑操作。我们使用 Roofline 性能模型分析了所得 TT-SVD 算法的性能。此外，我们展示了针对共享内存和分布式内存架构的不同算法变体的性能结果。我们的实验表明，常用的 TT-SVD 实现存在严重的性能损失。我们得出结论：一个专门用于张量分解核心的库将使社区受益：计算低秩近似的成本可以低至从主内存读取数据两次。因此，一个实现实际性能的实现将推动必须转向仅处理部分数据的随机化方法的极限。
英文摘要：
There are several factorizations of multi-dimensional tensors into lower-dimensional components, known as `tensor networks'. We consider the popular `tensor-train' (TT) format and ask: How efficiently can we compute a low-rank approximation from a full tensor on current multi-core CPUs? Compared to sparse and dense linear algebra, kernel libraries for multi-linear algebra are rare and typically not as well optimized. Linear algebra libraries like BLAS and LAPACK may provide the required operations in principle, but often at the cost of additional data movements for rearranging memory layouts. Furthermore, these libraries are typically optimized for the compute-bound case (e.g.\ square matrix operations) whereas low-rank tensor decompositions lead to memory bandwidth limited operations. We propose a `tensor-train singular value decomposition' (TT-SVD) algorithm based on two building blocks: a `Q-less tall-skinny QR' factorization, and a fused tall-skinny matrix-matrix multiplication and reshape operation. We analyze the performance of the resulting TT-SVD algorithm using the Roofline performance model. In addition, we present performance results for different algorithmic variants for shared-memory as well as distributed-memory architectures. Our experiments show that commonly used TT-SVD implementations suffer severe performance penalties. We conclude that a dedicated library for tensor factorization kernels would benefit the community: Computing a low-rank approximation can be as cheap as reading the data twice from main memory. As a consequence, an implementation that achieves realistic performance will move the limit at which one has to resort to randomized methods that only process part of the data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
