In this paper, we provide novel algorithms with identifiability guarantees for simplex-structured matrix factorization (SSMF), a generalization of nonnegative matrix factorization. Current state-of-the-art algorithms that provide identifiability results for SSMF rely on the sufficiently scattered condition (SSC) which requires the data points to be well spread within the convex hull of the basis vectors. The conditions under which our proposed algorithms recover the unique decomposition is in most cases much weaker than the SSC. We only require to have $d$ points on each facet of the convex hull of the basis vectors whose dimension is $d-1$. The key idea is based on extracting facets containing the largest number of points. We illustrate the effectiveness of our approach on synthetic data sets and hyperspectral images, showing that it outperforms state-of-the-art SSMF algorithms as it is able to handle higher noise levels, rank deficient matrices, outliers, and input data that highly violates the SSC.
This work focuses on the problem of unraveling nonlinearly mixed latent components in an unsupervised manner. The latent components are assumed to reside in the probability simplex, and are transformed by an unknown post-nonlinear mixing system. This problem finds various applications in signal and data analytics, e.g., nonlinear hyperspectral unmixing, image embedding, and nonlinear clustering. Linear mixture learning problems are already ill-posed, as identifiability of the target latent components is hard to establish in general. With unknown nonlinearity involved, the problem is even more …
Learning Causal Representations from General Environments: Identifiability and Intrinsic Ambiguity
Institute for Computational & Mathematical Engineering, Stanford University（斯坦福大学计算与数学工程研究所）；Department of Management Science & Engineering, Stanford University（斯坦福大学管理科学与工程系）
Oregon State Univ.（俄勒冈州立大学）；Google Inc.（谷歌公司）；IBM Research（IBM研究院）
Multiple views of data, both naturally acquired (e.g., image and audio) and artificially produced (e.g., via adding different noise to data samples), have proven useful in enhancing representation learning. Natural views are often handled by multiview analysis tools, e.g., (deep) canonical correlation analysis [(D)CCA], while the artificial ones are frequently used in self-supervised learning (SSL) paradigms, e.g., BYOL and Barlow Twins. Both types of approaches often involve learning neural feature extractors such that the embeddings of data exhibit high cross-view correlations. Although intu…
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of Cambridge（剑桥大学）
Independent component analysis provides a principled framework for unsupervised representation learning, with solid theory on the identifiability of the latent code that generated the data, given only observations of mixtures thereof. Unfortunately, when the mixing is nonlinear, the model is provably nonidentifiable, since statistical independence alone does not sufficiently constrain the problem. Identifiability can be recovered in settings where additional, typically observed variables are included in the generative process. We investigate an alternative path and consider instead including a…
Max Planck Institute for Intelligent Systems Tübingen（马克斯·普朗克智能系统研究所（蒂宾根））；University of Cambridge（剑桥大学）；Tübingen AI Center, University of Tübingen（蒂宾根大学蒂宾根人工智能中心）；IMPRS for Intelligent Systems（智能系统国际马克斯·普朗克研究学院）；Amazon（亚马逊）
Self-supervised representation learning has shown remarkable success in a number of domains. A common practice is to perform data augmentation via hand-crafted transformations intended to leave the semantics of the data invariant. We seek to understand the empirical success of this approach from a theoretical perspective. We formulate the augmentation process as a latent variable model by postulating a partition of the latent representation into a content component, which is assumed invariant to augmentation, and a style component, which is allowed to change. Unlike prior work on disentangleme…
DSReg: Provably Recovering Individual World Latents without Reconstruction
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Cold Spring Harbor Laboratory（冷泉港实验室）；Brown University（布朗大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；ELLIS Institute Tübingen（ELLIS图宾根研究所）
We revisit the problem of general identifiability originally introduced in [Lee et al., 2019] for causal inference and note that it is necessary to add positivity assumption of observational distribution to the original definition of the problem. We show that without such an assumption the rules of do-calculus and consequently the proposed algorithm in [Lee et al., 2019] are not sound. Moreover, adding the assumption will cause the completeness proof in [Lee et al., 2019] to fail. Under positivity assumption, we present a new algorithm that is provably both sound and complete. A nice property …
This work studies the robust subspace tracking (ST) problem. Robust ST can be simply understood as a (slow) time-varying subspace extension of robust PCA. It assumes that the true data lies in a low-dimensional subspace that is either fixed or changes slowly with time. The goal is to track the changing subspaces over time in the presence of additive sparse outliers and to do this quickly (with a short delay). We introduce a "fast" mini-batch robust ST solution that is provably correct under mild assumptions. Here "fast" means two things: (i) the subspace changes can be detected and the subspac…
Efficient Symbolic Computations for Identifying Causal Effects
Max Planck Institute for Mathematics in the Sciences（马克斯·普朗克数学研究院）；Binghamton University（宾夕法尼亚州立大学布林茅尔分校）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
Carnegie Mellon University（卡内基梅隆大学）；Microsoft Research, New York（微软研究院（纽约））；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Microsoft Research, Redmond（微软研究院（雷德蒙德））
We study the exploration problem in episodic MDPs with rich observations generated from a small number of latent states. Under certain identifiability assumptions, we demonstrate how to estimate a mapping from the observations to latent states inductively through a sequence of regression and clustering steps -- where previously decoded latent states provide labels for later regression problems -- and use it to construct good exploration policies. We provide finite-sample guarantees on the quality of the learned state decoding function and exploration policies, and complement our theory with an…
University of New South Wales（新南威尔士大学）；University of Sydney（悉尼大学）；Hong Kong Baptist University（香港浸会大学）；RIKEN AIP（日本理化学研究所创新智能综合研究中心）；University of Tokyo（东京大学）
In label-noise learning, the transition matrix plays a key role in building statistically consistent classifiers. Existing consistent estimators for the transition matrix have been developed by exploiting anchor points. However, the anchor-point assumption is not always satisfied in real scenarios. In this paper, we propose an end-to-end framework for solving label-noise learning without anchor points, in which we simultaneously optimize two objectives: the cross entropy loss between the noisy label and the predicted probability by the neural network, and the volume of the simplex formed by th…
Simplex Demixing: Disentangling Multiple Light-Flavor Jets at Colliders
Massachusetts Institute of Technology（麻省理工学院）；Center for Theoretical Physics -- a Leinweber Institute（理论物理中心——莱因韦伯研究所）；The NSF AI Institute for Artificial Intelligence and Fundamental Interactions（美国国家科学基金会人工智能与基本相互作用人工智能研究所）
Langevin Multiplicative Weights Update with Applications in Polynomial Portfolio Management
School of Information Management & Engineering, Shanghai University of Finance and Economics（上海财经大学信息管理与工程学院）；College of Business, Shanghai University of Finance and Economics（上海财经大学商学院）
Department of Technology, Management and Economics, Danish Technical University（丹麦技术大学技术、管理与经济学系）；Department of Mathematics and Statistics, Binghamton University, State University of New York（纽约州立大学宾汉姆顿分校数学与统计系）；Department of Mathematical Sciences, University of Copenhagen（哥本哈根大学数学科学系）
