半监督神经网络的截断变分EM
Truncated Variational EM for Semi-Supervised Neural Simpletrons
摘要：
概率生成网络的推理和学习通常非常具有挑战性，并且通常会阻止可扩展性与用于深度判别方法的大型网络一样。为了获得有效的可训练，大规模和性能良好的生成网络的半监督学习，我们在这里结合两个最新的发展：神经网络重新制定的分层泊松混合（神经Simpletrons），和一种新的截断变分EM方法（TV-EM）。TV-EM为生成网络中的学习提供了理论保证，其在神经Simpletron中的应用导致了学习方程的特别紧凑但近似最优的修改。如果应用于标准基准测试，我们根据经验发现，学习收敛于更少的EM迭代，每次EM迭代的复杂性降低，并且最终的似然值平均更高。对于具有少量标签的数据集的分类任务，如果与没有截断的应用程序相比，学习改进会导致错误率始终较低。本文中对MNIST数据集的实验允许在半监督设置中与标准和最先进的模型进行比较。NIST SD 19数据集上的进一步实验表明，当有大量额外的未标记数据可用时，该方法具有可扩展性。
英文摘要：
Inference and learning for probabilistic generative networks is often very challenging and typically prevents scalability to as large networks as used for deep discriminative approaches. To obtain efficiently trainable, large-scale and well performing generative networks for semi-supervised learning, we here combine two recent developments: a neural network reformulation of hierarchical Poisson mixtures (Neural Simpletrons), and a novel truncated variational EM approach (TV-EM). TV-EM provides theoretical guarantees for learning in generative networks, and its application to Neural Simpletrons results in particularly compact, yet approximately optimal, modifications of learning equations. If applied to standard benchmarks, we empirically find, that learning converges in fewer EM iterations, that the complexity per EM iteration is reduced, and that final likelihood values are higher on average. For the task of classification on data sets with few labels, learning improvements result in consistently lower error rates if compared to applications without truncation. Experiments on the MNIST data set herein allow for comparison to standard and state-of-the-art models in the semi-supervised setting. Further experiments on the NIST SD19 data set show the scalability of the approach when a manifold of additional unlabeled data is available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
