首页/论文/2011.06070

arXiv 2011.060702020-11cs.LG

Quantifying and Learning Linear Symmetry-Based Disentanglement

Eindhoven University of Technology (TU/e)（埃因霍温理工大学）；Eindhoven Artificial Intelligence Systems Institute (EAISI)（埃因霍温人工智能系统研究所）；Prosus（Prosus公司）

Loek Tonnaer, Luis A. Pérez Rey, Vlado Menkovski, Mike Holenderski, Jacobus W. Portegies

备注：: The Thirty-ninth International Conference on Machine Learning (ICML 2022)

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

The definition of Linear Symmetry-Based Disentanglement (LSBD) formalizes the notion of linearly disentangled representations, but there is currently no metric to quantify LSBD. Such a metric is crucial to evaluate LSBD methods and to compare to previous understandings of disentanglement. We propose $\mathcal{D}_\mathrm{LSBD}$, a mathematically sound metric to quantify LSBD, and provide a practical implementation for $\mathrm{SO}(2)$ groups. Furthermore, from this metric we derive LSBD-VAE, a semi-supervised method to learn LSBD representations. We demonstrate the utility of our metric by showing that (1) common VAE-based disentanglement methods don't learn LSBD representations, (2) LSBD-VAE as well as other recent methods can learn LSBD representations, needing only limited supervision on transformations, and (3) various desirable properties expressed by existing disentanglement metrics are also achieved by LSBD representations.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Quantifying the Effects of Enforcing Disentanglement on Variational Autoencoders

University of Cambridge（剑桥大学）

The notion of disentangled autoencoders was proposed as an extension to the variational autoencoder by introducing a disentanglement parameter $β$, controlling the learning pressure put on the possible underlying latent representations. For certain values of $β$ this kind of autoencoders is capable of encoding independent input generative factors in separate elements of the code, leading to a more interpretable and predictable model behaviour. In this paper we quantify the effects of the parameter $β$ on the model performance and disentanglement. After training multiple models with the same va…

2017-11-24cs.LGstat.ML

CFASL：用于变分自编码器解耦的复合因子对齐对称性学习

CFASL: Composite Factor-Aligned Symmetry Learning for Disentanglement in Variational AutoEncoder

AI Graduate School（人工智能研究生院）；Gwangju Institute of Science and Technology (GIST)（光州科学技术院）

提出CFASL方法，通过引入对称性码本、复合对称性和群等变编解码器，在无需数据集因子信息的无监督学习下实现VAE的解耦，并显著提升单因子与多因子变化条件下的解耦性能。

2024-11-12cs.AIcs.LG

A Metric for Linear Symmetry-Based Disentanglement

Eindhoven University of Technology（埃因霍温理工大学）；Prosus

The definition of Linear Symmetry-Based Disentanglement (LSBD) proposed by (Higgins et al., 2018) outlines the properties that should characterize a disentangled representation that captures the symmetries of data. However, it is not clear how to measure the degree to which a data representation fulfills these properties. We propose a metric for the evaluation of the level of LSBD that a data representation achieves. We provide a practical method to evaluate this metric and use it to evaluate the disentanglement of the data representations obtained for three datasets with underlying $SO(2)$ sy…

2020-11-26cs.LG

纠正常见解耦度量中的缺陷

Correcting Flaws in Common Disentanglement Metrics

University of Edinburgh（爱丁堡大学）；University of Oxford（牛津大学）；Beihang University（北京航空航天大学）；TU Wien（维也纳工业大学）

本文指出现有解耦度量存在两个缺陷，并提出两个新度量标准予以纠正，同时将组合泛化视为分类问题以衡量编码器的解耦能力，发现新度量与该任务表现相关性最强。

2023-04-05cs.LG

Surrogate Model For Field Optimization Using Beta-VAE Based Regression

Visage Technology

Oilfield development related decisions are made using reservoir simulation-based optimization study in which different production scenarios and well controls are compared. Such simulations are computationally expensive and so surrogate models are used to accelerate studies. Deep learning has been used in past to generate surrogates, but such models often fail to quantify prediction uncertainty and are not interpretable. In this work, beta-VAE based regression is proposed to generate simulation surrogates for use in optimization workflow. beta-VAE enables interpretable, factorized representatio…

2022-02-27cs.LGstat.ML

Evaluating the Disentanglement of Deep Generative Models through Manifold Topology

Stanford University（斯坦福大学）

Learning disentangled representations is regarded as a fundamental task for improving the generalization, robustness, and interpretability of generative models. However, measuring disentanglement has been challenging and inconsistent, often dependent on an ad-hoc external model or specific to a certain dataset. To address this, we present a method for quantifying disentanglement that only uses the generative model, by measuring the topological similarity of conditional submanifolds in the learned representation. This method showcases both unsupervised and supervised variants. To illustrate the…

2021-03-17cs.CVcs.LGstat.ML

更多相关论文收起更多论文

迈向一种改进的解耦表示评估指标

Towards an Improved Metric for Evaluating Disentangled Representations

University of Passau（帕绍大学）

本文系统比较现有解耦表示评估指标，提出基于排他性和改进因子-代码关系的新指标 EDI，实验表明其更稳定且能衡量核心性质，可作为标准化评估方法。

2024-10-04cs.AIcs.LG

DCI-ES: An Extended Disentanglement Framework with Connections to Identifiability

Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；School of Informatics, University of Edinburgh（爱丁堡大学信息学院）；Department of Engineering, University of Cambridge（剑桥大学工程系）；Technical University of Denmark（丹麦技术大学）

In representation learning, a common approach is to seek representations which disentangle the underlying factors of variation. Eastwood & Williams (2018) proposed three metrics for quantifying the quality of such disentangled representations: disentanglement (D), completeness (C) and informativeness (I). In this work, we first connect this DCI framework to two common notions of linear and nonlinear identifiability, thereby establishing a formal link between disentanglement and the closely-related field of independent component analysis. We then propose an extended DCI-ES framework with two ne…

2023-02-16cs.AIcs.CVcs.LG

On the non-universality of deep learning: quantifying the cost of symmetry

EPFL（洛桑联邦理工学院）；MIT（麻省理工学院）

We prove limitations on what neural networks trained by noisy gradient descent (GD) can efficiently learn. Our results apply whenever GD training is equivariant, which holds for many standard architectures and initializations. As applications, (i) we characterize the functions that fully-connected networks can weak-learn on the binary hypercube and unit sphere, demonstrating that depth-2 is as powerful as any other depth for this task; (ii) we extend the merged-staircase necessity result for learning with latent low-dimensional structure [ABM22] to beyond the mean-field regime. Under cryptogra…

2022-10-14cs.LG

通过无监督对称群发现实现解缠表示学习

Disentangled Representation Learning through Unsupervised Symmetry Group Discovery

Sorbonne Université, CNRS, Institut des Systèmes Intelligents et de Robotique, ISIR（索邦大学，国家科学研究中心，智能系统与机器人研究所，ISIR）

提出一种具身智能体通过与环境的无监督交互自主发现动作空间的群结构的方法，证明了在最小假设下真实对称群分解的可识别性，并推导出两种算法以学习线性对称基解缠表示。

2026-05-25cs.LG

Learning Group Structure and Disentangled Representations of Dynamical Environments

École Normale Supérieure（巴黎高等师范学院）；University of Oxford（牛津大学）

Learning disentangled representations is a key step towards effectively discovering and modelling the underlying structure of environments. In the natural sciences, physics has found great success by describing the universe in terms of symmetry preserving transformations. Inspired by this formalism, we propose a framework, built upon the theory of group representation, for learning representations of a dynamical environment structured around the transformations that generate its evolution. Experimentally, we learn the structure of explicitly symmetric environments without supervision from obse…

2020-10-25cs.LGstat.ML

Linear Disentangled Representations and Unsupervised Action Estimation

Southampton University（南安普顿大学）

Disentangled representation learning has seen a surge in interest over recent times, generally focusing on new models which optimise one of many disparate disentanglement metrics. Symmetry Based Disentangled Representation learning introduced a robust mathematical framework that defined precisely what is meant by a "linear disentangled representation". This framework determined that such representations would depend on a particular decomposition of the symmetry group acting on the data, showing that actions would manifest through irreducible group representations acting on independent represen…

2020-12-15cs.LGstat.ML

原子基础模型中的信息路由：任务对齐与等变性如何塑造线性解缠

Information Routing in Atomistic Foundation Models: How Task Alignment and Equivariance Shape Linear Disentanglement

Independent Researcher（独立研究者）

研究通过CPD方法揭示原子基础模型中任务对齐与等变性如何影响线性解缠，发现任务对齐主导几何信息可访问性，且对称类型影响信息路由。

2026-03-09cs.AIcs.LGphysics.chem-ph