Positive Semidefinite Matrix Factorization: A Connection with Phase Retrieval and Affine Rank Minimization
Tel Aviv University（特拉维夫大学）；University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）；IRIT（图卢兹计算机科学研究所）；Université de Toulouse（图卢兹大学）；CNRS（法国国家科学研究中心）；National University of Singapore（新加坡国立大学）
Dana Lahat, Yanbin Lang, Vincent Y. F. Tan, Cédric Févotte
备注：
18 pages (16 paper + 2 supplementary material), 9 figures, accepted for publication in the IEEE Transactions on Signal Processing. This is a revised version: there is a new additional PSDMF algorithm based on CGIHT, more numerical experiments, and some background material moved to Supplementary Material (pages 17 and 18 in this document). Supplementary Material also contains some extra figures
Positive semidefinite matrix factorization (PSDMF) expresses each entry of a nonnegative matrix as the inner product of two positive semidefinite (psd) matrices. When all these psd matrices are constrained to be diagonal, this model is equivalent to nonnegative matrix factorization. Applications include combinatorial optimization, quantum-based statistical models, and recommender systems, among others. However, despite the increasing interest in PSDMF, only a few PSDMF algorithms were proposed in the literature. In this work, we provide a collection of tools for PSDMF, by showing that PSDMF algorithms can be designed based on phase retrieval (PR) and affine rank minimization (ARM) algorithms. This procedure allows a shortcut in designing new PSDMF algorithms, as it allows to leverage some of the useful numerical properties of existing PR and ARM methods to the PSDMF framework. Motivated by this idea, we introduce a new family of PSDMF algorithms based on iterative hard thresholding (IHT). This family subsumes previously-proposed projected gradient PSDMF methods. We show that there is high variability among PSDMF optimization problems that makes it beneficial to try a number of methods based on different principles to tackle difficult problems. In certain cases, our proposed methods are the only algorithms able to find a solution. In certain other cases, they converge faster. Our results support our claim that the PSDMF framework can inherit desired numerical properties from PR and ARM algorithms, leading to more efficient PSDMF algorithms, and motivate further study of the links between these models.
A Non-commutative Extension of Lee-Seung's Algorithm for Positive Semidefinite Factorizations
National University of Singapore（新加坡国立大学）；Institute of High Performance Computing, Agency for Science, Technology and Research（新加坡科技研究局高性能计算研究所）；Singapore University of Technology and Design（新加坡科技设计大学）
矩阵分解是研究梯度下降隐式正则化的一个简单而自然的测试平台。Gunasekar et al.（2017）证明了具有无穷小初始化的梯度流收敛于最小化核范数的解，但最近的一系列论文认为范数最小化的语言不足以给出隐式正则化的完整表征。在这项工作中，我们提供了理论和经验证据，证明对于深度2矩阵分解，具有无穷小初始化的梯度流在数学上等价于一个简单的启发式秩最小化算法，贪婪低秩学习，在一些合理的假设下。这将先前工作中的秩最小化观点推广到更广泛的环境，并使我们能够构建反例来反驳Gunasekar等人的猜想。我们还将结果扩展到深度为$\ge 3$的情况，并且我们表明更深的好处是上述收敛对初始化幅度的依赖性要弱得多，因此这种秩最小化更有可能对实际规模的初始化生效。
Multiplicative updates for symmetric-cone factorizations
National University of Singapore（新加坡国立大学）；Institute of High Performance Computing, Agency for Science, Technology and Research（新加坡科技研究局高性能计算研究院）；Singapore University of Technology and Design（新加坡科技设计大学）
A Generative Word Embedding Model and its Low Rank Positive Semidefinite Solution
Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly (LILY)（南洋理工大学-不列颠哥伦比亚大学活跃老年生活卓越联合研究中心（LILY））；Nanyang Technological University（南洋理工大学）；Tsinghua University（清华大学）
