Generative neural networks have a well recognized ability to estimate underlying manifold structure of high dimensional data. However, if a single latent space is used, it is not possible to faithfully represent a manifold with topology different from Euclidean space. In this work we define the general class of Atlas Generative Models (AGMs), models with hybrid discrete-continuous latent space that estimate an atlas on the underlying data manifold together with a partition of unity on the data space. We identify existing examples of models from various popular generative paradigms that fit into this class. Due to the atlas interpretation, ideas from non-linear latent space analysis and statistics, e.g. geodesic interpolation, which has previously only been investigated for models with simply connected latent spaces, may be extended to the entire class of AGMs in a natural way. We exemplify this by generalizing an algorithm for graph based geodesic interpolation to the setting of AGMs, and verify its performance experimentally.
Interpolating between points is a problem connected simultaneously with finding geodesics and study of generative models. In the case of geodesics, we search for the curves with the shortest length, while in the case of generative models we typically apply linear interpolation in the latent space. However, this interpolation uses implicitly the fact that Gaussian is unimodal. Thus the problem of interpolating in the case when the latent density is non-Gaussian is an open problem. In this paper, we present a general and unified approach to interpolation, which simultaneously allows us to search…
Ohio University（俄亥俄大学）；Duke University（杜克大学）；University of Kentucky（肯塔基大学）
In this paper, we propose a nonlinear distance metric learning scheme based on the fusion of component linear metrics. Instead of merging displacements at each data point, our model calculates the velocities induced by the component transformations, via a geodesic interpolation on a Lie transfor- mation group. Such velocities are later summed up to produce a global transformation that is guaranteed to be diffeomorphic. Consequently, pair-wise distances computed this way conform to a smooth and spatially varying metric, which can greatly benefit k-NN classification. Experiments on synthetic and…
Interpolation for Robust Learning: Data Augmentation on Wasserstein Geodesics
Carnegie Mellon University（卡内基梅隆大学）；AT&T Chief Data Office（AT&T首席数据办公室）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Michigan（密歇根大学）
Enforcing Latent Euclidean Geometry in Single-Cell VAEs for Manifold Interpolation
Institute of Computational Biology, Helmholtz Munich, Munich, Germany（海德堡医学院计算生物学研究所）；Lamin Labs（Lamin实验室）；Munich Data Science Institute, Technical University of Munich, Germany（慕尼黑数据科学研究所）；TUM School of Life Sciences Weihenstephan, Technical University of Munich, Germany（技术大学生命科学学院）
Gatsby Computational Neuroscience Unit, University College London（伦敦大学学院盖茨比计算神经科学单元）；Department of Mathematics and Center for Scientific Computation and Mathematical Modeling, University of Maryland College Park（马里兰大学帕克分校数学系与科学计算与数学建模中心）
While variational autoencoders have been successful in several tasks, the use of conventional priors are limited in their ability to encode the underlying structure of input data. We introduce an Encoded Prior Sliced Wasserstein AutoEncoder wherein an additional prior-encoder network learns an embedding of the data manifold which preserves topological and geometric properties of the data, thus improving the structure of latent space. The autoencoder and prior-encoder networks are iteratively trained using the Sliced Wasserstein distance. The effectiveness of the learned manifold encoding is ex…
Exploiting the deep generative model's remarkable ability of learning the data-manifold structure, some recent researches proposed a geometric data interpolation method based on the geodesic curves on the learned data-manifold. However, this interpolation method often gives poor results due to a topological difference between the model and the dataset. The model defines a family of simply-connected manifolds, whereas the dataset generally contains disconnected regions or holes that make them non-simply-connected. To compensate this difference, we propose a novel density regularizer that make t…
Technical University of Munich（慕尼黑工业大学）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
The underlying geometrical structure of the latent space in deep generative models is in most cases not Euclidean, which may lead to biases when comparing interpolation capabilities of two models. Smoothness and plausibility of linear interpolations in latent space are associated with the quality of the underlying generative model. In this paper, we show that not all such interpolations are comparable as they can deviate arbitrarily from the shortest interpolation curve given by the geodesic. This deviation is revealed by computing curve lengths with the pull-back metric of the generative mode…
We introduce a Unified Disentanglement Network (UFDN) trained on The Cancer Genome Atlas (TCGA). We demonstrate that the UFDN learns a biologically relevant latent space of gene expression data by applying our network to two classification tasks of cancer status and cancer type. Our UFDN specific algorithms perform comparably to random forest methods. The UFDN allows for continuous, partial interpolation between distinct cancer types. Furthermore, we perform an analysis of differentially expressed genes between skin cutaneous melanoma(SKCM) samples and the same samples interpolated into gliobl…
Learning Lineage-guided Geodesics with Finsler Geometry
New York Genome Center（纽约基因组中心）；Irving Institute for Cancer Dynamics, Columbia University（癌症动力学研究所，哥伦比亚大学）；Department of Systems Biology, Columbia University（系统生物学系，哥伦比亚大学）；Department of Computer Science, Columbia University（计算机科学系，哥伦比亚大学）；Department of Biomedical Engineering, Columbia University（生物医学工程系，哥伦比亚大学）；Data Science Institute, Columbia University（数据科学研究所，哥伦比亚大学）
What's Inside Your Diffusion Model? A Score-Based Riemannian Metric to Explore the Data Manifold
Institut de la Vision & Laboratoire des Systèmes Perceptifs（视觉研究所及感知系统实验室）；Sorbonne Université & École Normale Supérieure（索邦大学及巴黎高等师范学院）；Group for Neural Theory（神经理论小组）；École Normale Supérieure（巴黎高等师范学院）
MolPIF: A Parameter Interpolation Flow Model for Molecule Generation
Lingang Laboratory（灵冈实验室）；School of Information Science and Technology（信息科学与技术学院）；ShanghaiTech University（上海科技大学）；Drug Discovery and Design Center, State Key Laboratory of Drug Research（药物发现与设计中心、国家药物研究重点实验室）；Shanghai Institute of Materia Medica, Chinese Academy of Sciences（中国科学院上海 medicinal materials 研究院）；Global Institute of Future Technology（未来技术全球研究院）；Shanghai Jiao Tong University（上海交通大学）；College of Computer Science and Artificial Intelligence（计算机科学与人工智能学院）
DTU Compute, Technical University of Denmark, Kongens Lyngby, Denmark（丹麦技术大学计算机系）；Department of Computer Science, University of Virginia, Charlottesville (VA), USA（弗吉尼亚大学计算机科学系）；Department of Electrical and Computer Engineering, University of Virginia, Charlottesville (VA), USA（弗吉尼亚大学电气与计算机工程系）
