软深度玻尔兹曼机
Soft-Deep Boltzmann Machines
摘要：
我们提出了一个分层玻尔兹曼机（BM），可以更好地利用分布式表示的优势。人们普遍认为，深BM（DBM）具有比其浅对应物，限制玻尔兹曼机（RBM）更大的代表性能力。然而，这种关于DBM优于RBM的期望从未在理论上得到验证。在本文中，我们提供了理论和经验证据，DBMs的代表性的权力，实际上是相当有限的，在利用分布式表示。我们提出了一个近似的衡量代表性的权力BM关于分布式表示的效率。通过这种措施，我们发现了一个令人惊讶的事实，DBMs可以使分布式表示的效率低下。基于这些观察，我们提出了一种替代BM架构，我们称之为软深BM（sDBMs）。我们表明，sDBMs可以更有效地利用分布式表示的措施。实验表明，sDBMs优于几个国家的最先进的模型，包括DBMs，在生成任务的二进制MNIST和加州理工学院-101轮廓。
英文摘要：
We present a layered Boltzmann machine (BM) that can better exploit the advantages of a distributed representation. It is widely believed that deep BMs (DBMs) have far greater representational power than its shallow counterpart, restricted Boltzmann machines (RBMs). However, this expectation on the supremacy of DBMs over RBMs has not ever been validated in a theoretical fashion. In this paper, we provide both theoretical and empirical evidences that the representational power of DBMs can be actually rather limited in taking advantages of distributed representations. We propose an approximate measure for the representational power of a BM regarding to the efficiency of a distributed representation. With this measure, we show a surprising fact that DBMs can make inefficient use of distributed representations. Based on these observations, we propose an alternative BM architecture, which we dub soft-deep BMs (sDBMs). We show that sDBMs can more efficiently exploit the distributed representations in terms of the measure. Experiments demonstrate that sDBMs outperform several state-of-the-art models, including DBMs, in generative tasks on binarized MNIST and Caltech-101 silhouettes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
