Deep Learning (DL) has attracted a lot of attention for its ability to reach state-of-the-art performance in many machine learning tasks. The core principle of DL methods consists in training composite architectures in an end-to-end fashion, where inputs are associated with outputs trained to optimize an objective function. Because of their compositional nature, DL architectures naturally exhibit several intermediate representations of the inputs, which belong to so-called latent spaces. When treated individually, these intermediate representations are most of the time unconstrained during the learning process, as it is unclear which properties should be favored. However, when processing a batch of inputs concurrently, the corresponding set of intermediate representations exhibit relations (what we call a geometry) on which desired properties can be sought. In this work, we show that it is possible to introduce constraints on these latent geometries to address various problems. In more details, we propose to represent geometries by constructing similarity graphs from the intermediate representations obtained when processing a batch of inputs. By constraining these Latent Geometry Graphs (LGGs), we address the three following problems: i) Reproducing the behavior of a teacher architecture is achieved by mimicking its geometry, ii) Designing efficient embeddings for classification is achieved by targeting specific geometries, and iii) Robustness to deviations on inputs is achieved via enforcing smooth variation of geometry between consecutive latent spaces. Using standard vision benchmarks, we demonstrate the ability of the proposed geometry-based methods in solving the considered problems.
High-Dimensional and Incomplete matrices, which usually contain a large amount of valuable latent information, can be well represented by a Latent Factor Analysis model. The performance of an LFA model heavily rely on its optimization process. Thereby, some prior studies employ the Particle Swarm Optimization to enhance an LFA model's optimization process. However, the particles within the swarm follow the static evolution paths and only share the global best information, which limits the particles' searching area to cause sub-optimum issue. To address this issue, this paper proposes a Dynamic…
Deep generative networks have been widely used for learning mappings from a low-dimensional latent space to a high-dimensional data space. In many cases, data transformations are defined by linear paths in this latent space. However, the Euclidean structure of the latent space may be a poor match for the underlying latent structure in the data. In this work, we incorporate a generative manifold model into the latent space of an autoencoder in order to learn the low-dimensional manifold structure from the data and adapt the latent space to accommodate this structure. In particular, we focus on …
Southwest University（西南大学）；University of Technology Sydney（悉尼科技大学）；Zhejiang Wanli University（浙江万里学院）
High-Dimensional and Incomplete (HDI) data are frequently found in various industrial applications with complex interactions among numerous nodes, which are commonly non-negative for representing the inherent non-negativity of node interactions. A Non-negative Latent Factor (NLF) model is able to extract intrinsic features from such data efficiently. However, existing NLF models all adopt a static divergence metric like Euclidean distance or α-\b{eta} divergence to build its learning objective, which greatly restricts its scalability of accurately representing HDI data from different domains. …
HiLNO: A Hierarchical Latent Neural Operator with Multi-Scale Supervision for PDEs on General Geometries
School of Mathematics, Nanjing University of Aeronautics and Astronautics（南京航空航天大学数学学院）；Key Laboratory of Mathematical Modelling and High Performance Computing of Air Vehicles (NUAA), MIIT（工业和信息化部南京航空航天大学数学建模与高性能计算重点实验室（NUAA））；School of Mathematics and Information Sciences, Yantai University（烟台大学数学与信息科学学院）
Dyson School of Design Engineering, Imperial College London（伦敦帝国理工学院戴森设计工程学院）；Impression Technologies Ltd（印象科技有限公司）
The latest sheet stamping processes enable efficient manufacturing of complex shape structural components that have high stiffness to weight ratios, but these processes can introduce defects. To assist component design for stamping processes, this paper presents a novel deep-learning-based platform for optimising 3D component geometries. The platform adopts a non-parametric modelling approach that is capable of optimising arbitrary geometries from multiple geometric parameterisation schema. This approach features the interaction of two neural networks: 1) a geometry generator and 2) a manufact…
Massachusetts Institute of Technology（麻省理工学院）；Harvard University（哈佛大学）
Many datasets represent a combination of different ways of looking at the same data that lead to different generalizations. For example, a corpus with examples generated by different people may be mixtures of many perspectives and can be viewed with different perspectives by others. It isnt always possible to represent the viewpoints by a clean separation, in advance, of examples representing each viewpoint and train a separate model for each viewpoint. We introduce lensing, a mixed initiative technique to extract lenses or mappings between machine learned representations and perspectives of h…
Recent work on the representation of functions on sets has considered the use of summation in a latent space to enforce permutation invariance. In particular, it has been conjectured that the dimension of this latent space may remain fixed as the cardinality of the sets under consideration increases. However, we demonstrate that the analysis leading to this conjecture requires mappings which are highly discontinuous and argue that this is only of limited practical use. Motivated by this observation, we prove that an implementation of this model via continuous mappings (as provided by e.g. neur…
Technical University of Denmark（丹麦技术大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；IT University of Copenhagen（哥本哈根信息技术大学）
Latent space geometry has shown itself to provide a rich and rigorous framework for interacting with the latent variables of deep generative models. The existing theory, however, relies on the decoder being a Gaussian distribution as its simple reparametrization allows us to interpret the generating process as a random projection of a deterministic manifold. Consequently, this approach breaks down when applied to decoders that are not as easily reparametrized. We here propose to use the Fisher-Rao metric associated with the space of decoder distributions as a reference metric, which we pull ba…
Geodesic Calculus on Implicitly Defined Latent Manifolds
University of Bonn, Germany（波恩大学，德国）；Université Paris–Saclay, ENS Paris–Saclay, Centre Borelli, France（巴黎-萨克莱大学，巴黎-萨克莱高等师范学院，Borelli研究中心，法国）；University of Münster, Germany（穆尔斯特大学，德国）
University of Waterloo（滑铁卢大学）；University of Notre Dame（圣母大学）
The problem of combined state and input estimation of linear structural systems based on measured responses and a priori knowledge of structural model is considered. A novel methodology using Gaussian process latent force models is proposed to tackle the problem in a stochastic setting. Gaussian process latent force models (GPLFMs) are hybrid models that combine differential equations representing a physical system with data-driven non-parametric Gaussian process models. In this work, the unknown input forces acting on a structure are modelled as Gaussian processes with some chosen covariance …
TU Berlin（柏林工业大学）；UC San Diego（加州大学圣地亚哥分校）；MIT（麻省理工学院）
We present a method to learn compositional multi-object dynamics models from image observations based on implicit object encoders, Neural Radiance Fields (NeRFs), and graph neural networks. NeRFs have become a popular choice for representing scenes due to their strong 3D prior. However, most NeRF approaches are trained on a single scene, representing the whole scene with a global model, making generalization to novel scenes, containing different numbers of objects, challenging. Instead, we present a compositional, object-centric auto-encoder framework that maps multiple views of the scene to a…
College of Liberal Arts and Sciences, National University of Defense Technology（国防科技大学文理学院）
Embedding real-world networks presents challenges because it is not clear how to identify their latent geometries. Embedding some disassortative networks, such as scale-free networks, to the Euclidean space has been shown to incur distortions. Embedding scale-free networks to hyperbolic spaces offer an exciting alternative but incurs distortions when embedding assortative networks with latent geometries not hyperbolic. We propose an inductive model that leverages both the expressiveness of GCNs and trivial bundle to learn inductive node representations for networks with or without node feature…
Understanding Latent Diffusability via Fisher Geometry
School of Mathematics, University of Minnesota（明尼苏达大学数学系）；NVIDIA（NVIDIA公司）；Department of Mathematics, The Ohio State University（俄亥俄州立大学数学系）；Zu Chongzhi Center and DIRC, Duke Kunshan University（杜克-昆山大学祖冲之中心和DIRC）
We introduce a representation learning framework for spatial trajectories. We represent partial observations of trajectories as probability distributions in a learned latent space, which characterize the uncertainty about unobserved parts of the trajectory. Our framework allows us to obtain samples from a trajectory for any continuous point in time, both interpolating and extrapolating. Our flexible approach supports directly modifying specific attributes of a trajectory, such as its pace, as well as combining different partial observations into single representations. Experiments show our met…
Carnegie Mellon University（卡内基梅隆大学）；Northeastern University（东北大学）；Google Research（谷歌研究院）；University of California, San Diego（加利福尼亚大学圣迭戈分校）
We propose a deep generative model that performs typography analysis and font reconstruction by learning disentangled manifolds of both font style and character shape. Our approach enables us to massively scale up the number of character types we can effectively model compared to previous methods. Specifically, we infer separate latent variables representing character and font via a pair of inference networks which take as input sets of glyphs that either all share a character type, or belong to the same font. This design allows our model to generalize to characters that were not observed duri…
Beyond Gaussian Worlds: Latent Geometry Matters for JEPAs
Université Paris-Saclay（巴黎-萨克雷大学）；CNRS（法国国家科学研究中心）；ENS Paris-Saclay（巴黎-萨克雷高等师范学校）；Centre Borelli（Borelli中心）；Advanced Track and Trace；IIE, Facultad de Ingeniería, Universidad de la República（共和国大学工程学院IIE）；Institut Universitaire de France（法国大学研究院）
GeoFunFlow: Geometric function flow matching for joint probabilistic inference of physical fields and complex geometries
Institute for Foundations of Data Science（数据科学基础研究所）；Yale University（耶鲁大学）；Department of Computer Science（计算机科学系）；Department of Internal Medicine（内科医学系）；Department of Statistics and Data Science（统计学与数据科学系）
This paper uses Factored Latent Analysis (FLA) to learn a factorized, segmental representation for observations of tracked objects over time. Factored Latent Analysis is latent class analysis in which the observation space is subdivided and each aspect of the original space is represented by a separate latent class model. One could simply treat these factors as completely independent and ignore their interdependencies or one could concatenate them together and attempt to learn latent class structure for the complete observation space. Alternatively, FLA allows the interdependencies to be explo…
Deep generative models are universal tools for learning data distributions on high dimensional data spaces via a mapping to lower dimensional latent spaces. We provide a study of latent space geometries and extend and build upon previous results on Riemannian metrics. We show how a class of heuristic measures gives more flexibility in finding meaningful, problem-specific distances, and how it can be applied to diverse generator types such as autoregressive generators commonly used in e.g. language and other sequence modeling. We further demonstrate how a diffusion-inspired transformation previ…
Universite de Montreal（蒙特利尔大学）；Penn State University（宾夕法尼亚州立大学）；McGill University（麦吉尔大学）
Advances in neural variational inference have facilitated the learning of powerful directed graphical models with continuous latent variables, such as variational autoencoders. The hope is that such models will learn to represent rich, multi-modal latent factors in real-world data, such as natural language text. However, current models often assume simplistic priors on the latent variables - such as the uni-modal Gaussian distribution - which are incapable of representing complex latent factors efficiently. To overcome this restriction, we propose the simple, but highly flexible, piecewise con…
Duke University（杜克大学）；Oak Ridge National Laboratory（橡树岭国家实验室）
Disentangled learning representations have promising utility in many applications, but they currently suffer from serious reliability issues. We present Gaussian Channel Autoencoder (GCAE), a method which achieves reliable disentanglement via flexible density estimation of the latent space. GCAE avoids the curse of dimensionality of density estimation by disentangling subsets of its latent space with the Dual Total Correlation (DTC) metric, thereby representing its high-dimensional latent joint distribution as a collection of many low-dimensional conditional distributions. In our experiments, …
Variational Rank Reduction Autoencoders for Generative Thermal Design
ESI Group-UZ Chair of the National Strategy on Artificial Intelligence（ESI集团-乌兹主席国家人工智能战略办公室）；Aragon Institute of Engineering Research (I3A)（阿拉贡工程研究所（I3A））；Universidad de Zaragoza（萨拉戈萨大学）；PIMM Lab（PIMM实验室）；ENSAM Institute of Technology（ENSAM技术研究所）
GALDS: A Graph-Autoencoder-based Latent Dynamics Surrogate model to predict neurite material transport
Department of Mechanical Engineering, Carnegie Mellon University（机械工程系，卡内基梅隆大学）；Department of Biomedical Engineering, Carnegie Mellon University（生物医学工程系，卡内基梅隆大学）；Department of Civil and Environmental Engineering, Carnegie Mellon University（土木与环境工程系，卡内基梅隆大学）
Spectral embedding of adjacency or Laplacian matrices of undirected graphs is a common technique for representing a network in a lower dimensional latent space, with optimal theoretical guarantees. The embedding can be used to estimate the community structure of the network, with strong consistency results in the stochastic blockmodel framework. One of the main practical limitations of standard algorithms for community detection from spectral embeddings is that the number of communities and the latent dimension of the embedding must be specified in advance. In this article, a novel Bayesian mo…
Arizona State University（亚利桑那州立大学）；Western Sydney University（西悉尼大学）
All reinforcement learning algorithms must handle the trade-off between exploration and exploitation. Many state-of-the-art deep reinforcement learning methods use noise in the action selection, such as Gaussian noise in policy gradient methods or $ε$-greedy in Q-learning. While these methods are appealing due to their simplicity, they do not explore the state space in a methodical manner. We present an approach that uses a model to derive reward bonuses as a means of intrinsic motivation to improve model-free reinforcement learning. A key insight of our approach is that this dynamics model ca…
In this work we approach the task of learning multilingual word representations in an offline manner by fitting a generative latent variable model to a multilingual dictionary. We model equivalent words in different languages as different views of the same word generated by a common latent variable representing their latent lexical meaning. We explore the task of alignment by querying the fitted model for multilingual embeddings achieving competitive results across a variety of tasks. The proposed model is robust to noise in the embedding space making it a suitable method for distributed repre…
Empirical observations suggest that in practice, community membership does not completely explain the dependency between the edges of an observation graph. The residual dependence of the graph edges are modeled in this paper, to first order, by auxiliary node latent variables that affect the statistics of the graph edges but carry no information about the communities of interest. We then study community detection in graphs obeying the stochastic block model and censored block model with auxiliary latent variables. We analyze the conditions for exact recovery when these auxiliary latent variabl…
University of Toronto（多伦多大学）；Vector Institute for AI（矢量人工智能研究院）；Samsung AI Center Toronto（多伦多三星人工智能中心）；Samsung Research America, Inc.（美国三星研究公司）
Representing 3D shape is a fundamental problem in artificial intelligence, which has numerous applications within computer vision and graphics. One avenue that has recently begun to be explored is the use of latent representations of generative models. However, it remains an open problem to learn a generative model of shape that is interpretable and easily manipulated, particularly in the absence of supervised labels. In this paper, we propose an unsupervised approach to partitioning the latent space of a variational autoencoder for 3D point clouds in a natural way, using only geometric inform…
MIT-IBM Watson AI Lab, IBM Research（IBM研究院MIT-IBM沃森人工智能实验室）；IBM Research - Tokyo（IBM研究院（东京））；University of Tokyo（东京大学）；Queen’s University（女王大学）
Current domain-independent, classical planners require symbolic models of the problem domain and instance as input, resulting in a knowledge acquisition bottleneck. Meanwhile, although deep learning has achieved significant success in many fields, the knowledge is encoded in a subsymbolic representation which is incompatible with symbolic systems such as planners. We propose Latplan, an unsupervised architecture combining deep learning and classical planning. Given only an unlabeled set of image pairs showing a subset of transitions allowed in the environment (training inputs), Latplan learns …
Many important problems in the real world don't have unique solutions. It is thus important for machine learning models to be capable of proposing different plausible solutions with meaningful probability measures. In this work we introduce such a deep learning framework that learns the one-to-many mappings between the inputs and outputs, together with faithful uncertainty measures. We call our framework {\it modal uncertainty estimation} since we model the one-to-many mappings to be generated through a set of discrete latent variables, each representing a latent mode hypothesis that explains …
Technical University of Munich（慕尼黑工业大学）；Ca’ Foscari University of Venice（威尼斯大学）；USI University of Lugano（瑞士意大利语区大学（卢加诺大学））；NVIDIA（英伟达）；Johns Hopkins University（约翰斯·霍普金斯大学）；University of Oxford（牛津大学）
Graphs are a powerful tool for representing and analyzing unstructured, non-Euclidean data ubiquitous in the healthcare domain. Two prominent examples are molecule property prediction and brain connectome analysis. Importantly, recent works have shown that considering relationships between input data samples have a positive regularizing effect for the downstream task in healthcare applications. These relationships are naturally modeled by a (possibly unknown) graph structure between input samples. In this work, we propose Graph-in-Graph (GiG), a neural network architecture for protein classifi…
