arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1303.7461stat.MLcs.LGmath.PR
收藏

离散单元窄信度网络的泛逼近深度与误差

Universal Approximation Depth and Errors of Narrow Belief Networks with Discrete Units

Pennsylvania State University(宾夕法尼亚州立大学)

Guido F. Montúfar

摘要:

我们推广了最近的理论工作的最小层数的狭窄的深度信念网络,可以近似任何概率分布的状态,其可见的单位任意好。我们将二进制单位的设置(Sutskever和Hinton,2008; Le Roux和Bengio,2008,2010; Montúfar和Ay,2011)放宽到具有任意有限状态空间的单位,并将消失的近似误差放宽到任意近似误差容限。例如,我们证明了对于某些$m\in \mathbb{N}$,具有宽度为$n \leq m + \log_q(m) + 1$的$L\geq 2+\frac{q^{\lceil m-δ\rceil}-1}{q-1}$层的$q$ -ary深度信念网络可以近似$\{0,1,\ldots,q-1\}^n$上的任何概率分布,而不会超过$δ$的库尔贝克-莱布勒散度。我们的分析包括离散限制玻尔兹曼机和朴素贝叶斯模型作为特殊情况。

英文摘要:

We generalize recent theoretical work on the minimal number of layers of narrow deep belief networks that can approximate any probability distribution on the states of their visible units arbitrarily well. We relax the setting of binary units (Sutskever and Hinton, 2008; Le Roux and Bengio, 2008, 2010; Montúfar and Ay, 2011) to units with arbitrary finite state spaces, and the vanishing approximation error to an arbitrary approximation error tolerance. For example, we show that a $q$-ary deep belief network with $L\geq 2+\frac{q^{\lceil m-δ\rceil}-1}{q-1}$ layers of width $n \leq m + \log_q(m) + 1$ for some $m\in \mathbb{N}$ can approximate any probability distribution on $\{0,1,\ldots,q-1\}^n$ without exceeding a Kullback-Leibler divergence of $δ$. Our analysis covers discrete restricted Boltzmann machines and naïve Bayes models as special cases.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑