To ensure interpretability of extracted sources in tensor decomposition, we introduce in this paper a dictionary-based tensor canonical polyadic decomposition which enforces one factor to belong exactly to a known dictionary. A new formulation of sparse coding is proposed which enables high dimensional tensors dictionary-based canonical polyadic decomposition. The benefits of using a dictionary in tensor decomposition models are explored both in terms of parameter identifiability and estimation accuracy. Performances of the proposed algorithms are evaluated on the decomposition of simulated data and the unmixing of hyperspectral images.
This work proposes to learn fair low-rank tensor decompositions by regularizing the Canonical Polyadic Decomposition factorization with the kernel Hilbert-Schmidt independence criterion (KHSIC). It is shown, theoretically and empirically, that a small KHSIC between a latent factor and the sensitive features guarantees approximate statistical parity. The proposed algorithm surpasses the state-of-the-art algorithm, FATR (Zhu et al., 2018), in controlling the trade-off between fairness and residual fit on synthetic and real data sets.
Computer Science Research Institute, Sandia National Laboratories（桑迪亚国家实验室计算机科学研究所）；Department of Statistics & Data Science, University of California, Los Angeles（加州大学洛杉矶分校统计与数据科学系）
ETH Zurich, Swiss Federal Institute of Technology（苏黎世联邦理工学院）
Acoustic monitoring for machine fault detection is a recent and expanding research path that has already provided promising results for industries. However, it is impossible to collect enough data to learn all types of faults from a machine. Thus, new algorithms, trained using data from healthy conditions only, were developed to perform unsupervised anomaly detection. A key issue in the development of these algorithms is the noise in the signals, as it impacts the anomaly detection performance. In this work, we propose a powerful data-driven and quasi non-parametric denoising strategy for spec…
Constrained tensor and matrix factorization models allow to extract interpretable patterns from multiway data. Therefore identifiability properties and efficient algorithms for constrained low-rank approximations are nowadays important research topics. This work deals with columns of factor matrices of a low-rank approximation being sparse in a known and possibly overcomplete basis, a model coined as Dictionary-based Low-Rank Approximation (DLRA). While earlier contributions focused on finding factor columns inside a dictionary of candidate columns, i.e. one-sparse approximations, this work is…
Joint blind source separation (J-BSS) is an emerging data-driven technique for multi-set data-fusion. In this paper, J-BSS is addressed from a tensorial perspective. We show how, by using second-order multi-set statistics in J-BSS, a specific double coupled canonical polyadic decomposition (DC-CPD) problem can be formulated. We propose an algebraic DC-CPD algorithm based on a coupled rank-1 detection mapping. This algorithm converts a possibly underdetermined DC-CPD to a set of overdetermined CPDs. The latter can be solved algebraically via a generalized eigenvalue decomposition based scheme. …
Osaka University（大阪大学）；The University of Tokyo（东京大学）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；Nara Institute of Science and Technology（奈良先端科学技术大学院大学）
Methods based on vector embeddings of knowledge graphs have been actively pursued as a promising approach to knowledge graph completion.However, embedding models generate storage-inefficient representations, particularly when the number of entities and relations, and the dimensionality of the real-valued embedding vectors are large. We present a binarized CANDECOMP/PARAFAC(CP) decomposition algorithm, which we refer to as B-CP, where real-valued parameters are replaced by binary values to reduce model size. Moreover, we show that a fast score computation technique can be developed with bitwise…
Efficient modelling of feature interactions underpins supervised learning for non-sequential tasks, characterized by a lack of inherent ordering of features (variables). The brute force approach of learning a parameter for each interaction of every order comes at an exponential computational and memory cost (Curse of Dimensionality). To alleviate this issue, it has been proposed to implicitly represent the model parameters as a tensor, the order of which is equal to the number of features; for efficiency, it can be further factorized into a compact Tensor Train (TT) format. However, both TT an…
Université de Mons（蒙斯大学）；Univ. Côte d’Azur（蔚蓝海岸大学）；CNRS（法国国家科学研究中心）；I3S（索菲亚安提波利斯信息学、信号与系统研究所）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Grenoble INP（格勒诺布尔国立理工学院）；GIPSA-lab（格勒诺布尔图像、语音信号与自动化实验室）
We propose an extension of the canonical polyadic (CP) tensor model where one of the latent factors is allowed to vary through data slices in a constrained way. The components of the latent factors, which we want to retrieve from data, can vary from one slice to another up to a diffeomorphism. We suppose that the diffeomorphisms are also unknown, thus merging curve registration and tensor decomposition in one model, which we call registered CP. We present an algorithm to retrieve both the latent factors and the diffeomorphism, which is assumed to be in a parametrized form. At the end of the pa…
Tensor decompositions are powerful tools for large data analytics as they jointly model multiple aspects of data into one framework and enable the discovery of the latent structures and higher-order correlations within the data. One of the most widely studied and used decompositions, especially in data mining and machine learning, is the Canonical Polyadic or CP decomposition. However, today's datasets are not static and these datasets often dynamically growing and changing with time. To operate on such large data, we present OCTen the first ever compression-based online parallel implementatio…
Sandia National Laboratories（桑迪亚国家实验室）；University of Michigan（密歇根大学）
Tensor decomposition is a well-known tool for multiway data analysis. This work proposes using stochastic gradients for efficient generalized canonical polyadic (GCP) tensor decomposition of large-scale tensors. GCP tensor decomposition is a recently proposed version of tensor decomposition that allows for a variety of loss functions such as Bernoulli loss for binary data or Huber loss for robust estimation. The stochastic gradient is formed from randomly sampled elements of the tensor and is efficient because it can be computed using the sparse matricized-tensor-times-Khatri-Rao product (MTTK…
Department of Statistics, University of Wisconsin-Madison（威斯康星大学麦迪逊分校统计学系）；Departments of Biostatistics & Bioinformatics and Computer Science, Duke University（杜克大学生物统计学与生物信息学系、计算机科学系）
University of Southern California（南加州大学）；Princeton University（普林斯顿大学）；University of Minnesota（明尼苏达大学）
We consider the problem of factorizing a structured 3-way tensor into its constituent Canonical Polyadic (CP) factors. This decomposition, which can be viewed as a generalization of singular value decomposition (SVD) for tensors, reveals how the tensor dimensions (features) interact with each other. However, since the factors are a priori unknown, the corresponding optimization problems are inherently non-convex. The existing guaranteed algorithms which handle this non-convexity incur an irreducible error (bias), and only apply to cases where all factors have the same structure. To this end, w…
Shenzhen Research Institute of Big Data（深圳大数据研究院）；Oregon State University（俄勒冈州立大学）；University of Minnesota（明尼苏达大学）
This work considers low-rank canonical polyadic decomposition (CPD) under a class of non-Euclidean loss functions that frequently arise in statistical machine learning and signal processing. These loss functions are often used for certain types of tensor data, e.g., count and binary tensors, where the least squares loss is considered unnatural.Compared to the least squares loss, the non-Euclidean losses are generally more challenging to handle. Non-Euclidean CPD has attracted considerable interests and a number of prior works exist. However, pressing computational and theoretical challenges, s…
给定过完备字典$A$和作为$A$的几个线性独立列的线性组合的信号$b$，经典稀疏恢复理论处理恢复唯一稀疏表示$x$使得$b = A x$的问题。已知在一定条件下，$A$上的$x$可以通过基追踪（BP）和正交匹配追踪（OMP）算法恢复。在这项工作中，我们考虑更一般的情况下，$b$在于一个低维子空间跨越的$A$，这可能是线性相关的一些列。在这种情况下，稀疏解$x$一般是不唯一的，我们研究的问题，表示$x$确定的子空间，即非零条目的$x$对应的字典原子的子空间。这样的表示$x$称为子空间稀疏。我们提出了保证子空间稀疏恢复的充分条件，这些条件具有清晰的几何解释，并解释了子空间稀疏恢复的性质。我们还证明了在随机化模型下，充分条件可以满足。我们的结果是适用于传统的稀疏恢复问题，我们得到的条件稀疏恢复的限制比规范的相互相干条件。我们还利用这些结果对基于稀疏表示的分类方法进行了分析，得到了证明其正确性的条件。
Tensor decomposition models play an increasingly important role in modern data science applications. One problem of particular interest is fitting a low-rank Canonical Polyadic (CP) tensor decomposition model when the tensor has sparse structure and the tensor elements are nonnegative count data. SparTen is a high-performance C++ library which computes a low-rank decomposition using different solvers: a first-order quasi-Newton or a second-order damped Newton method, along with the appropriate choice of runtime parameters. Since default parameters in SparTen are tuned to experimental results i…
Institute of Information Theory and Automation（信息与自动化研究所）；Brain Science Institute, RIKEN（理化学研究所脑科学研究所）；Systems Research Institute, Polish Academy of Sciences（波兰科学院系统研究所）
