Tensor networks are a class of algorithms aimed at reducing the computational complexity of high-dimensional problems. They are used in an increasing number of applications, from quantum simulations to machine learning. Exploiting data parallelism in these algorithms is key to using modern hardware. However, there are several ways to map required tensor operations onto linear algebra routines ("building blocks"). Optimizing this mapping impacts the numerical behavior, so computational and numerical aspects must be considered hand-in-hand. In this paper we discuss the performance of solvers for low-rank linear systems in the tensor-train format (also known as matrix-product states). We consider three popular algorithms: TT-GMRES, MALS, and AMEn. We illustrate their computational complexity based on the example of discretizing a simple high-dimensional PDE in, e.g., $50^{10}$ grid points. This shows that the projection to smaller sub-problems for MALS and AMEn reduces the number of floating-point operations by orders of magnitude. We suggest optimizations regarding orthogonalization steps, singular value decompositions, and tensor contractions. In addition, we propose a generic preconditioner based on a TT-rank-1 approximation of the linear operator. Overall, we obtain roughly a 5x speedup over the reference algorithm for the fastest method (AMEn) on a current multicore CPU.
A User Manual for cuHALLaR: A GPU Accelerated Low-Rank Semidefinite Programming Solver
H. M. Stewart School of Industrial and Systems Engineering, Georgia Tech（H.M. 斯图尔特工业与系统工程学院，佐治亚理工学院）；School of Applied Mathematics, FGV（应用数学学院，FGV）；Department of Combinatorics and Optimization, University of Waterloo（组合优化系，滑铁卢大学）
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究所）；Moscow Institute of Physics and Technology（莫斯科物理技术学院）；Institute of Numerical Mathematics of Russian Academy of Sciences（俄罗斯科学院计算数学研究所）
Mixed-Precision in adaptive Runge-Kutta method for large ODE systems
Univ. Rennes（雷恩大学）；Inria Lyon（法国国家信息与自动化研究所里昂分部）；CNRS, Université Claude Bernard Lyon 1（法国国家科学研究中心，克莱贝尔·贝尔纳·里昂第一大学）；INSA de Lyon, Université Claude Bernard Lyon 1（里昂国立应用科学学院，克莱贝尔·贝尔纳·里昂第一大学）；INSA Lyon（里昂国立应用科学学院）
