arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2207.06137stat.MLcs.AIcs.LG
收藏

独立机制分析对表示学习鲁棒性的探讨

Probing the Robustness of Independent Mechanism Analysis for Representation Learning

Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Eberhard Karls University of Tübingen(埃伯哈德·卡尔斯·图宾根大学);Swiss Federal Institute of Technology Zürich(苏黎世联邦理工学院);University of Cambridge(剑桥大学)

Joanna Sliwa, Shubhangi Ghosh, Vincent Stimper, Luigi Gresele, Bernhard Schölkopf

摘要:

表示学习的一个目标是恢复生成数据的原始潜在代码,这是一项需要额外信息或归纳偏差的任务。最近提出的一种称为独立机制分析(IMA)的方法假设,每个潜在的源应该独立地影响所观察到的混合物,补充标准的非线性独立成分分析,并从独立的因果机制的原则的灵感。虽然在理论和实验中表明,IMA有助于恢复真实的潜伏期,但该方法的性能到目前为止仅在建模假设完全满足时才被表征。在这里,我们测试的方法的鲁棒性违反的基本假设。我们发现,基于IMA的正则化的好处,恢复真正的源扩展到混合函数与不同程度的违反IMA原则,而标准的正则化不提供相同的优点。此外,我们表明,非正则化的极大似然恢复混合函数,系统地偏离IMA原则,并提供了一个参数阐明IMA为基础的正则化的好处。

英文摘要:

One aim of representation learning is to recover the original latent code that generated the data, a task which requires additional information or inductive biases. A recently proposed approach termed Independent Mechanism Analysis (IMA) postulates that each latent source should influence the observed mixtures independently, complementing standard nonlinear independent component analysis, and taking inspiration from the principle of independent causal mechanisms. While it was shown in theory and experiments that IMA helps recovering the true latents, the method's performance was so far only characterized when the modeling assumptions are exactly satisfied. Here, we test the method's robustness to violations of the underlying assumptions. We find that the benefits of IMA-based regularization for recovering the true sources extend to mixing functions with various degrees of violation of the IMA principle, while standard regularizers do not provide the same merits. Moreover, we show that unregularized maximum likelihood recovers mixing functions which systematically deviate from the IMA principle, and provide an argument elucidating the benefits of IMA-based regularization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑