带隐藏单元的网络推理
Network Inference with Hidden Units
摘要:
我们推导出学习规则,从记录的历史的一个“可见的”子集的单位,在随机动态网络中找到单位之间的连接。我们考虑两个模型。在这两种方法中,可见单位都是二元的和随机的。在一个模型中,“隐藏”单元是连续值的,具有S形激活函数,而在另一个模型中,它们是二元的,像可见的一样随机。我们推导出这两种情况下的精确学习规则。对于随机的情况下,执行精确的计算需要,在一般情况下,重复求和的配置数量的系统的大小和数据长度呈指数增长,这是不可行的大型系统。我们推导出一个平均场论,基于一个因式分解的分析隐藏单元状态的分布,这提供了一个有吸引力的替代大系统。我们提出了一些数值计算的结果,说明了两种模型的主要特点,随机的情况下,精确和近似计算。
英文摘要:
We derive learning rules for finding the connections between units in stochastic dynamical networks from the recorded history of a ``visible'' subset of the units. We consider two models. In both of them, the visible units are binary and stochastic. In one model the ``hidden'' units are continuous-valued, with sigmoidal activation functions, and in the other they are binary and stochastic like the visible ones. We derive exact learning rules for both cases. For the stochastic case, performing the exact calculation requires, in general, repeated summations over an number of configurations that grows exponentially with the size of the system and the data length, which is not feasible for large systems. We derive a mean field theory, based on a factorized ansatz for the distribution of hidden-unit states, which offers an attractive alternative for large systems. We present the results of some numerical calculations that illustrate key features of the two models and, for the stochastic case, the exact and approximate calculations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。