数据不完整的问题，即，在生物医学信号处理、网络流量分析、文献计量学、社会网络分析、化学计量学、计算机视觉、通信网络等中普遍存在具有缺失值或未知值的数据。我们考虑如何以捕获数据的潜在结构并可能重构缺失值（即，张量完备化）。我们专注于一个最著名的张量分解，捕捉多线性结构，CANDECOMP/PARAFAC（CP）。在存在缺失数据的情况下，CP可以被公式化为仅对已知条目建模的加权最小二乘问题。我们开发了一种算法称为CP-WOPT（CP加权优化），使用一阶优化方法来解决加权最小二乘问题。基于大量的数值实验，我们的算法被证明是成功的因式分解张量与噪声和高达99%的缺失数据。我们的方法的一个独特之处在于它可以扩展到稀疏的大规模数据，例如，1000 x 1000 x 1000，500万个已知条目（密度为0.5%）。我们进一步证明了CP-WOPT在两个现实世界的应用程序的有用性：一个新的EEG（脑电图）应用程序，经常遇到的数据丢失，由于断开电极和建模的计算机网络流量的数据可能会缺席的问题，由于数据收集过程的费用。
英文摘要：
The problem of incomplete data - i.e., data with missing or unknown values - in multi-way arrays is ubiquitous in biomedical signal processing, network traffic analysis, bibliometrics, social network analysis, chemometrics, computer vision, communication networks, etc. We consider the problem of how to factorize data sets with missing values with the goal of capturing the underlying latent structure of the data and possibly reconstructing missing values (i.e., tensor completion). We focus on one of the most well-known tensor factorizations that captures multi-linear structure, CANDECOMP/PARAFAC (CP). In the presence of missing data, CP can be formulated as a weighted least squares problem that models only the known entries. We develop an algorithm called CP-WOPT (CP Weighted OPTimization) that uses a first-order optimization approach to solve the weighted least squares problem. Based on extensive numerical experiments, our algorithm is shown to successfully factorize tensors with noise and up to 99% missing data. A unique aspect of our approach is that it scales to sparse large-scale data, e.g., 1000 x 1000 x 1000 with five million known entries (0.5% dense). We further demonstrate the usefulness of CP-WOPT on two real-world applications: a novel EEG (electroencephalogram) application where missing data is frequently encountered due to disconnections of electrodes and the problem of modeling computer network traffic where data may be absent due to the expense of the data collection process.
Many application problems that lead to solving linear systems make use of preconditioned Krylov subspace solvers to compute their solution. Among the most popular preconditioning approaches are incomplete factorization methods either as single-level approaches or within a multilevel framework. We will present a block incomplete factorization that is based on skillfully blocking the system initially and throughout the factorization. This approach allows for the use of cache-optimized dense matrix kernels such as level-3 BLAS or LAPACK. We will demonstrate how this block approach outperforms the…
A hybrid recursive multilevel incomplete factorization preconditioner for solving general linear systems
Johann Bernoulli Institute for Mathematics and Computer Science - University of Groningen（格罗宁根大学）；School of Mathematical Sciences, University of Electronic Science and Technology of China（电子科技大学）
SYM-ILDL: Incomplete $LDL^{T}$ Factorization of Symmetric Indefinite and Skew-Symmetric Matrices
SYM-ILDL是一个数值软件包，用于计算对称不定和实反对称矩阵的不完全$LDL^{T}$（或“ILDL”）分解。该算法的核心是不完全LU（ILU）的Crout变体，最初由Li和Saad针对对称矩阵引入和实现[Crout versions of ILU factorization with pivoting for sparse symmetric matrices，Transactions on数值分析20，pp. 75- 85，2005年]。我们的代码是经济的存储方面，它处理真正的反对称矩阵以及，除了对称的。该软件包是用C++编写的，它是模板化的，开源的，并包括一个MATLAB接口。该代码包括内置的RCM和AMD重排序，两个平衡策略，阈值Bunch-Kaufman旋转和rook旋转，以及一个MC 64的包装器，一个流行的基于匹配的平衡和重排序算法。我们还包括两个内置的迭代求解器：SQMR预处理与ILDL，或MINRES预处理与对称正定预条件的基础上ILDL因式分解。
We introduce a software package called HIFIR for preconditioning sparse, unsymmetric, ill-conditioned, and potentially singular systems. HIFIR computes a hybrid incomplete factorization, which combines multilevel incomplete LU factorization with a truncated, rank-revealing QR factorization on the final Schur complement. This novel hybridization is based on the new theory of approximate generalized inverse and $ε$-accuracy. It enables near-optimal preconditioners for consistent systems and enables flexible GMRES to solve inconsistent systems when coupled with iterative refinement. In this paper…
Mixed precision multigrid with smoothing based on incomplete Cholesky factorization
CNRS-IRIT（法国国家科学研究中心-信息学与系统技术研究所）；IFP Energies Nouvelles（法国石油研究院）；Department of Numerical Mathematics, Charles University（布拉格查理大学数值数学系）；Technical University of Munich - Campus Heilbronn（慕尼黑工业大学海尔布隆校区）；Ludwig-Maximilians-Universität München - Department of Earth and Environmental Sciences（慕尼黑路德维希-马克西米利安大学地球与环境科学系）；Department of Computer Science, FAU Erlangen-Nürnberg（埃尔朗根-纽伦堡大学计算机系）；Department of Applied Mathematics, VSB-Technical University of Ostrava（俄斯特拉发理工大学应用数学系）；Centre Européen de Recherche et de Formation Avancée en Calcul Scientifique（欧洲高级科学计算研究与培训中心）
Row-Splitting ILU Preconditioners for Sparse Least-Squares Problems
STFC Rutherford Appleton Laboratory（STFC卢瑟福阿普尔顿实验室）；School of Mathematical, Physical and Computational Sciences, University of Reading（雷丁大学数学、物理与计算科学学院）；Department of Numerical Mathematics, Faculty of Mathematics and Physics, Charles University（查理大学数学物理学院数值数学系）
Scalable methods for nonnegative matrix factorizations of near-separable tall-and-skinny matrices
Stanford University（斯坦福大学）；Purdue University（普渡大学）；Purdue University Institute for Computational and Bindley Biosciences Center（普渡大学计算与Bindley生物科学中心）；Computer Science Mathematical Engineering（计算机科学数学工程）
Stanford University（斯坦福大学）；Duke University（杜克大学）；International Computer Science Institute（国际计算机科学研究所）；University of California, Berkeley（加州大学伯克利分校）
Kernel methods are widespread in machine learning; however, they are limited by the quadratic complexity of the construction, application, and storage of kernel matrices. Low-rank matrix approximation algorithms are widely used to address this problem and reduce the arithmetic and storage cost. However, we observed that for some datasets with wide intra-class variability, the optimal kernel parameter for smaller classes yields a matrix that is less well approximated by low-rank methods. In this paper, we propose an efficient structured low-rank approximation method -- the Block Basis Factoriza…
Factorization of large dense matrices are ubiquitous in engineering and data science applications, e.g. preconditioners for iterative boundary integral solvers, frontal matrices in sparse multifrontal solvers, and computing the determinant of covariance matrices. HSS and $\mathcal{H}^2$-matrices are hierarchical low-rank matrix formats that can reduce the complexity of factorizing such dense matrices from $\mathcal{O}(N^3)$ to $\mathcal{O}(N)$. For HSS matrices, it is possible to remove the dependency on the trailing matrices during Cholesky/LU factorization, which results in a highly parallel…
We present a study of the effectiveness of asynchronous incomplete LU factorization preconditioners for the time-implicit solution of compressible flow problems while exploiting thread-parallelism within a compute node. A block variant of the asynchronous fine-grain parallel preconditioner adapted to a finite volume discretization of the compressible Navier-Stokes equations on unstructured grids is presented, and convergence theory is extended to the new variant. Experimental (numerical) results on the performance of these preconditioners on inviscid and viscous laminar two-dimensional steady-…
Incomplete factorization is a widely used preconditioning technique for Krylov subspace methods for solving large-scale sparse linear systems. Its multilevel variants, such as ILUPACK, are more robust for many symmetric or unsymmetric linear systems than the traditional, single-level incomplete LU (or ILU) techniques. However, the previous multilevel ILU techniques still lacked robustness and efficiency for some large-scale saddle-point problems, which often arise from systems of partial differential equations (PDEs). We introduce HILUCSI, or Hierarchical Incomplete LU-Crout with Scalability-o…
VeST: Very Sparse Tucker Factorization of Large-Scale Tensors
Daegu Gyeongbuk Institute of Science and Technology（大邱庆北科学技术院）；Department of Computer Science and Engineering, Seoul National University（首尔大学计算机科学与工程系）
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；IQVIA（艾昆纬）
Low-rank tensor factorization or completion is well-studied and applied in various online settings, such as online tensor factorization (where the temporal mode grows) and online tensor completion (where incomplete slices arrive gradually). However, in many real-world settings, tensors may have more complex evolving patterns: (i) one or more modes can grow; (ii) missing entries may be filled; (iii) existing tensor elements can change. Existing methods cannot support such complex scenarios. To fill the gap, this paper proposes a Generalized Online Canonical Polyadic (CP) Tensor factorization an…
Solving systems of linear equations is a problem occuring frequently in water engineering applications. Usually the size of the problem is too large to be solved via direct factorization. One can resort to iterative approaches, in particular the conjugate gradients method if the matrix is symmetric positive definite. Preconditioners further enhance the rate of convergence but hitherto only handcrafted ones requiring expert knowledge have been used. We propose an innovative approach employing Machine Learning, in particular a Convolutional Neural Network, to unassistedly design preconditioning …
