张量训练算法的并行算法
Parallel Algorithms for Tensor Train Arithmetic
摘要：
我们提出了高效的和可扩展的并行算法进行数学运算的低秩张量表示的张量训练（TT）格式。我们考虑加法，逐元素乘法，计算范数和内积，正交化和舍入（秩截断）的算法。这些是应用程序的核心操作，例如利用TT结构的迭代Krylov求解器。并行算法是为分布式内存计算而设计的，我们使用数据分布和策略，在TT格式内为各个核心并行计算。我们分析了所提出算法的计算和通信成本以显示其可扩展性，并且我们提供了数值实验来证明它们在共享内存和分布式内存并行系统上的效率。例如，我们在舍入2GB TT张量时观察到比现有MATLAB TT-100更好的单核性能，并且我们的实现使用单个节点的所有40个核心实现了$34\times$加速。我们还展示了在所有数学运算中，在高达10，000多个核心的较大TT张量上的近线性并行缩放。
英文摘要：
We present efficient and scalable parallel algorithms for performing mathematical operations for low-rank tensors represented in the tensor train (TT) format. We consider algorithms for addition, elementwise multiplication, computing norms and inner products, orthogonalization, and rounding (rank truncation). These are the kernel operations for applications such as iterative Krylov solvers that exploit the TT structure. The parallel algorithms are designed for distributed-memory computation, and we use a data distribution and strategy that parallelizes computations for individual cores within the TT format. We analyze the computation and communication costs of the proposed algorithms to show their scalability, and we present numerical experiments that demonstrate their efficiency on both shared-memory and distributed-memory parallel systems. For example, we observe better single-core performance than the existing MATLAB TT-Toolbox in rounding a 2GB TT tensor, and our implementation achieves a $34\times$ speedup using all 40 cores of a single node. We also show nearly linear parallel scaling on larger TT tensors up to over 10,000 cores for all mathematical operations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
