arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1809.01804cs.LGstat.ML
收藏

在变分自编码器中发现影响因素

Discovering Influential Factors in Variational Autoencoders

Xi'an Jiaotong University(西安交通大学);Brunel University London(布鲁内尔大学伦敦分校);State University of New York at Buffalo(纽约州立大学布法罗分校)

Shiqi Liu, Jingxin Liu, Qian Zhao, Xiangyong Cao, Huibin Li, Deyu Meng, Hongying Meng, Sheng Liu

AI总结:

本文提出通过互信息指标在变分自编码器中识别影响因素,通过实验验证其在数据重建和分类任务中的有效性。

摘要:

在机器学习领域,仍然存在一个关键问题,即在不手动干预或依赖直觉帮助提取有用知识或服务于下游任务的情况下,如何识别和监督学习到的表示。在本工作中,我们关注监督由变分自编码器(VAE)提取的影响因素。VAE被提出以在面对有时预设因素被忽略的问题时学习独立的低维表示。我们认为输入和每个学习到的表示因素之间的互信息是发现影响因素的必要指标。我们发现VAE的目标倾向于在因子维度上诱导互信息稀疏性,相对于数据内在维度,从而导致一些非影响因素,其在数据重建中的功能可以被忽略。我们还表明互信息也影响VAE重建误差的下界和下游分类任务。为了使该指标具有应用性,我们设计了一个算法来计算VAE的互信息,并证明其一致性。在MNIST、CelebA和DEAP数据集上的实验结果表明,互信息可以帮助确定影响因素,其中一些是可解释的,可用于进一步的生成和分类任务,并帮助在DEAP数据集上发现与情绪相关的变量。

英文摘要:

In the field of machine learning, it is still a critical issue to identify and supervise the learned representation without manually intervening or intuition assistance to extract useful knowledge or serve for the downstream tasks. In this work, we focus on supervising the influential factors extracted by the variational autoencoder(VAE). The VAE is proposed to learn independent low dimension representation while facing the problem that sometimes pre-set factors are ignored. We argue that the mutual information of the input and each learned factor of the representation plays a necessary indicator of discovering the influential factors. We find the VAE objective inclines to induce mutual information sparsity in factor dimension over the data intrinsic dimension and therefore result in some non-influential factors whose function on data reconstruction could be ignored. We show mutual information also influences the lower bound of the VAE's reconstruction error and downstream classification task. To make such indicator applicable, we design an algorithm for calculating the mutual information for the VAE and prove its consistency. Experimental results on MNIST, CelebA and DEAP datasets show that mutual information can help determine influential factors, of which some are interpretable and can be used to further generation and classification tasks, and help discover the variant that connects with emotion on DEAP dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑