The recent low-rank prior based models solve the tensor completion problem efficiently. However, these models fail to exploit the local patterns of tensors, which compromises the performance of tensor completion. In this paper, we propose a novel hierarchical prior regularized matrix factorization model for tensor completion. This model hierarchically incorporates the low-rank prior, total variation prior, and sparse coding prior into a matrix factorization, simultaneously characterizing both the global low-rank property and the local smoothness of tensors. For solving the proposed model, we use the alternating direction method of multipliers to establish our algorithm. Besides, the complexity and convergence are investigated to further validate the algorithm effectiveness. The proposed scheme is then evaluated through various data sets. Experiment results verify that, the proposed method outperforms several state-of-the-art approaches.
In this work, we focus on connections between $K$-means clustering approaches and Orthogonal Nonnegative Matrix Factorization (ONMF) methods. We present a novel framework to extract the distance measure and the centroids of the $K$-means method based on first order conditions of the considered ONMF objective function, which exploits the classical alternating minimization schemes of Nonnegative Matrix Factorization (NMF) algorithms. While this technique is characterized by a simple derivation procedure, it can also be applied to non-standard regularized ONMF models. Using this framework, we con…
We present a general-purpose data compression algorithm, Regularized L21 Semi-NonNegative Matrix Factorization (L21 SNF). L21 SNF provides robust, parts-based compression applicable to mixed-sign data for which high fidelity, individualdata point reconstruction is paramount. We derive a rigorous proof of convergenceof our algorithm. Through experiments, we show the use-case advantages presentedby L21 SNF, including application to the compression of highly overdeterminedsystems encountered broadly across many general machine learning processes.
Tensor decomposition is a popular technique for tensor completion, However most of the existing methods are based on linear or shallow model, when the data tensor becomes large and the observation data is very small, it is prone to over fitting and the performance decreases significantly. To address this problem, the completion method for a tensor based on a Biased Deep Tensor Factorization Network (BDTFN) is proposed. This method can not only overcome the shortcomings of traditional tensor factorization, but also deal with complex non-linear data. Firstly, the horizontal and lateral tensors c…
Fast and Accurate Tensor Completion with Total Variation Regularized Tensor Trains
Department of Electrical and Electronic Engineering, The University of Hong Kong（香港大学电子工程系）；Delft Center for Systems and Control, Delft University of Technology（代尔夫特理工大学系统与控制中心）
Guangdong University of Technology（广东工业大学）；RIKEN（理化学研究所）
For the high dimensional data representation, nonnegative tensor ring (NTR) decomposition equipped with manifold learning has become a promising model to exploit the multi-dimensional structure and extract the feature from tensor data. However, the existing methods such as graph regularized tensor ring decomposition (GNTR) only models the pair-wise similarities of objects. For tensor data with complex manifold structure, the graph can not exactly construct similarity relationships. In this paper, in order to effectively utilize the higher-dimensional and complicated similarities among objects,…
Alternating minimization algorithms for graph regularized tensor completion
Huawei European Research Institute（华为欧洲研究院）；LISN, INRIA, Université Paris-Saclay（法国国家信息与自动化研究所，巴黎萨克雷大学）；LSEC, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院科学与工程计算国家重点实验室）；Department of INMA, Université catholique de Louvain（鲁汶天主教大学）
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；IQVIA（艾昆纬）
Low-rank tensor factorization or completion is well-studied and applied in various online settings, such as online tensor factorization (where the temporal mode grows) and online tensor completion (where incomplete slices arrive gradually). However, in many real-world settings, tensors may have more complex evolving patterns: (i) one or more modes can grow; (ii) missing entries may be filled; (iii) existing tensor elements can change. Existing methods cannot support such complex scenarios. To fill the gap, this paper proposes a Generalized Online Canonical Polyadic (CP) Tensor factorization an…
If learning methods are to scale to the massive sizes of modern datasets, it is essential for the field of machine learning to embrace parallel and distributed computing. Inspired by the recent development of matrix factorization methods with rich theory but poor computational complexity and by the relative ease of mapping matrices onto distributed architectures, we introduce a scalable divide-and-conquer framework for noisy matrix factorization. We present a thorough theoretical analysis of this framework in which we characterize the statistical errors introduced by the "divide" step and cont…
This paper introduces an algorithm for the nonnegative matrix factorization-and-completion problem, which aims to find nonnegative low-rank matrices X and Y so that the product XY approximates a nonnegative data matrix M whose elements are partially known (to a certain accuracy). This problem aggregates two existing problems: (i) nonnegative matrix factorization where all entries of M are given, and (ii) low-rank matrix completion where nonnegativity is not required. By taking the advantages of both nonnegativity and low-rankness, one can generally obtain superior results than those of just us…
Central China Normal University（华中师范大学）；The University of Hong Kong（香港大学）
In this paper, we study the sparse nonnegative tensor factorization and completion problem from partial and noisy observations for third-order tensors. Because of sparsity and nonnegativity, the underlying tensor is decomposed into the tensor-tensor product of one sparse nonnegative tensor and one nonnegative tensor. We propose to minimize the sum of the maximum likelihood estimation for the observations with nonnegativity constraints and the tensor $\ell_0$ norm for the sparse factor. We show that the error bounds of the estimator of the proposed model can be established under general noise o…
Matrix factorization is a popular framework for modeling low-rank data matrices. Motivated by manifold learning problems, this paper proposes a quadratic matrix factorization (QMF) framework to learn the curved manifold on which the dataset lies. Unlike local linear methods such as the local principal component analysis, QMF can better exploit the curved structure of the underlying manifold. Algorithmically, we propose an alternating minimization algorithm to optimize QMF and establish its theoretical convergence properties. Moreover, to avoid possible over-fitting, we then propose a regulariz…
In order to solve large matrix completion problems with practical computational cost, an approximate approach based on matrix factorization has been widely used. Alternating least squares (ALS) and stochastic gradient descent (SGD) are two major algorithms to this end. In this study, we propose a new algorithm, namely cavity-based matrix factorization (CBMF) and approximate cavity-based matrix factorization (ACBMF), which are developed based on the cavity method from statistical mechanics. ALS yields solutions with less iterations when compared to those of SGD. This is because its update rules…
Xiamen University（厦门大学）；University of Science and Technology of China（中国科学技术大学）；Hong Kong University of Science and Technology（香港科技大学）；Xiamen University of Technology（厦门理工学院）
We introduce two data completion algorithms for the limited-aperture problems in inverse acoustic scattering. Both completion algorithms are independent of the topological and physical properties of the unknown scatterers. The main idea is to relate the limited-aperture data to the full-aperture data via the prolate matrix. The data completion algorithms are simple and fast since only the approximate inversion of the prolate matrix is involved. We then combine the data completion algorithms with imaging methods such as factorization method and direct sampling method for the object reconstructi…
