arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1702.08648cs.LG
收藏

Auto-clustering Output Layer:神经网络中潜在注释的自动学习

Auto-clustering Output Layer: Automatic Learning of Latent Annotations in Neural Networks

Ozsel Kilinc, Ismail Uysal

摘要:

在本文中,我们讨论了一种不同类型的半监督设置:对所有观测值都可以使用粗略的标记,但模型必须为每个观测值学习精细的潜在注释。在文本分类、蛋白质功能预测、图像分类以及医学和基因组学研究等探索性科学研究中,可能会遇到这种情况下的问题。我们认为这种设置是同时执行监督分类(根据可用的粗标签)和无监督聚类(在每个粗标签内),并提出了一种新的输出层修改称为自动聚类输出层(ACOL),允许并发分类和聚类基于图的活动正则化(GAR)技术。由于所提出的输出层修改在输出层为每个类复制softmax节点,因此GAR允许在传统纠错学习框架上的这些副本之间进行竞争学习,以最终使神经网络能够在这种部分监督设置中学习潜在注释。我们演示了粗糙标签监督如何影响性能,并有助于传播子类之间的有用聚类信息。三个最流行的图像数据集MNIST,SVHN和CIFAR-100的比较测试严格证明了所提出的方法的有效性和竞争力。

英文摘要:

In this paper, we discuss a different type of semi-supervised setting: a coarse level of labeling is available for all observations but the model has to learn a fine level of latent annotation for each one of them. Problems in this setting are likely to be encountered in many domains such as text categorization, protein function prediction, image classification as well as in exploratory scientific studies such as medical and genomics research. We consider this setting as simultaneously performed supervised classification (per the available coarse labels) and unsupervised clustering (within each one of the coarse labels) and propose a novel output layer modification called auto-clustering output layer (ACOL) that allows concurrent classification and clustering based on Graph-based Activity Regularization (GAR) technique. As the proposed output layer modification duplicates the softmax nodes at the output layer for each class, GAR allows for competitive learning between these duplicates on a traditional error-correction learning framework to ultimately enable a neural network to learn the latent annotations in this partially supervised setup. We demonstrate how the coarse label supervision impacts performance and helps propagate useful clustering information between sub-classes. Comparative tests on three of the most popular image datasets MNIST, SVHN and CIFAR-100 rigorously demonstrate the effectiveness and competitiveness of the proposed approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑