arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1702.08720stat.MLcs.LG
收藏

通过信息最大化自增强训练学习离散表示

Learning Discrete Representations via Information Maximizing Self-Augmented Training

University of Tokyo(东京大学);RIKEN AIP(日本理化学研究所先进智能研究中心);Preferred Networks, Inc.(Preferred Networks公司);ATR Cognitive Mechanism Laboratories(ATR认知机制实验室)

Weihua Hu, Takeru Miyato, Seiya Tokui, Eiichi Matsumoto, Masashi Sugiyama

摘要:

学习数据的离散表示是一个中心的机器学习任务,因为表示的紧凑性和易于解释。该任务包括聚类和哈希学习作为特殊情况。深度神经网络很有希望被使用,因为它们可以对数据的非线性进行建模并扩展到大型数据集。然而,它们的模型复杂性是巨大的,因此,我们需要仔细地正则化网络,以便学习有用的表示,这些表示对于感兴趣的应用表现出预期的不变性。为此,我们提出了一种称为信息最大化自增强训练(IMSAT)的方法。在IMSAT中,我们使用数据增强来对离散表示施加不变性。更具体地说,我们鼓励增强数据点的预测表示以端到端的方式接近原始数据点。与此同时,我们最大限度地提高了数据与其预测离散表示之间的信息理论依赖性。在基准数据集上进行的大量实验表明,IMSAT在聚类和无监督哈希学习方面都能产生最先进的结果。

英文摘要:

Learning discrete representations of data is a central machine learning task because of the compactness of the representations and ease of interpretation. The task includes clustering and hash learning as special cases. Deep neural networks are promising to be used because they can model the non-linearity of data and scale to large datasets. However, their model complexity is huge, and therefore, we need to carefully regularize the networks in order to learn useful representations that exhibit intended invariance for applications of interest. To this end, we propose a method called Information Maximizing Self-Augmented Training (IMSAT). In IMSAT, we use data augmentation to impose the invariance on discrete representations. More specifically, we encourage the predicted representations of augmented data points to be close to those of the original data points in an end-to-end fashion. At the same time, we maximize the information-theoretic dependency between data and their predicted discrete representations. Extensive experiments on benchmark datasets show that IMSAT produces state-of-the-art results for both clustering and unsupervised hash learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑