表示学习中协变量依赖与潜在结构之间的权衡

中文摘要 AI 辅助

解耦表示学习旨在获取其各个维度分别对应于不同协变量的潜在表示。无监督方法通常以潜在维度的独立性为目标，但这无法保证所得维度与语义有意义的协变量对齐。监督方法利用观测到的协变量构建潜在空间结构，但在协变量存在相关性时，它们无法同时控制潜在-协变量的一一对应对齐和潜在独立性。我们提出了一种统一的监督框架，该框架将潜在维度与协变量的依赖关系和潜在结构的约束相结合。在该框架内，我们证明存在一种固有权衡：强制潜在独立性或排他性的潜在-协变量一一对应对齐，会以潜在-协变量对齐的可证明损失为代价。我们证明所得的解耦机制按该对齐的强度排序，每种机制都允许对潜在空间进行闭式变换。我们将这些变换事后应用于重新对齐预训练模型（如CLIP、DINOv2和ViT）的表示，并将其融入知情因子分析（iFA，一种具有协变量知情因子的概率模型）的推理中。在模拟数据和真实多组学数据上，我们表明事后对齐和iFA都能实现结构化潜在表示的可控性。

英文摘要

Disentangled representation learning seeks latent representations whose indicidual dimensions each align with a distinct covariate. Unsupervised approaches typically target latent dimension independence, yet this gives no guarantee that the resulting dimensions align with semantically meaningful covariates. Supervised approaches structure the latent space using observed covariates, but under correlated covariates they cannot simultaneously control one-to-one latent-covariate alignment and latent independence. We introduce a unified, supervised framework that couples latent dimension-covariate dependence with constraints on the latent structure. Within this framework, we show an inherent trade-off, where enforcing latent independence or exclusive one-to-one latent-covariate dependence comes at a provable cost in latent-covariate alignment. We prove that the resulting disentanglement regimes are ordered by the strength of that alignment. Each regime admits a closed-form transformation of the latent space. We apply these transformations post-hoc to realign the representations of pretrained models such as CLIP, DINOv2, and ViT, and we fold them into the inference of informed factor analysis (iFA), a probabilistic model with covariate-informed factors. On simulated and real multi-omics data, we show that both post-hoc alignment and iFA enable controllability of structured latent representations.

发表机构

• Faculty of Mathematics, Informatics, and Mechanics, University of Warsaw（华沙大学数学、信息学与力学学院）
• Institute of AI for Health, Helmholtz Munich（亥姆霍兹慕尼黑健康人工智能研究所）

机构由 AI 辅助整理，请以论文原文为准。