arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1111.1020cs.ITmath.ITstat.ML
收藏

随机信念传播:一种低复杂度的和积算法

Stochastic Belief Propagation: A Low-Complexity Alternative to the Sum-Product Algorithm

Nima Noorshams, Martin J. Wainwright

摘要:

和积或置信传播(BP)算法是一种广泛使用的消息传递算法,用于计算具有离散变量的图形模型中的边际分布。在BP消息更新的核心处,当应用于具有成对交互的图形模型时,存在具有在状态维度$d$中为二次的复杂性的矩阵-向量积,并且需要将实数(消息)的$(d-1)$维向量传输到其邻居。由于各种应用涉及非常大的状态维度,因此这种计算和通信复杂性可能非常复杂。在本文中,我们提出了一个低复杂度的BP变种,称为随机信念传播(SBP)。顾名思义,它是BP消息更新的自适应随机版本,其中每个节点将随机选择的信息传递给每个邻居。SBP消息更新将$d$中的计算复杂性(每次迭代)从二次降低到线性,而不假设任何特定的势结构,并且还显著降低了通信复杂性,每个边缘仅需要$\log{d}$比特传输。此外,我们建立了一些理论保证SBP的性能,表明它几乎肯定收敛到BP不动点的任何树结构的图,并与周期满足收缩性条件的图。此外,对于这些图形模型,我们提供了非渐近上界的收敛速度,表明$\ell_{\infty}$规范的误差向量衰减不慢于$O(1/\sqrt{t})$与树的迭代次数$t$和均方误差衰减为$O(1/t)$一般的图形。这些分析表明,SBP可以证明降低计算和通信复杂性的各种类别的图形模型。

英文摘要:

The sum-product or belief propagation (BP) algorithm is a widely-used message-passing algorithm for computing marginal distributions in graphical models with discrete variables. At the core of the BP message updates, when applied to a graphical model with pairwise interactions, lies a matrix-vector product with complexity that is quadratic in the state dimension $d$, and requires transmission of a $(d-1)$-dimensional vector of real numbers (messages) to its neighbors. Since various applications involve very large state dimensions, such computation and communication complexities can be prohibitively complex. In this paper, we propose a low-complexity variant of BP, referred to as stochastic belief propagation (SBP). As suggested by the name, it is an adaptively randomized version of the BP message updates in which each node passes randomly chosen information to each of its neighbors. The SBP message updates reduce the computational complexity (per iteration) from quadratic to linear in $d$, without assuming any particular structure of the potentials, and also reduce the communication complexity significantly, requiring only $\log{d}$ bits transmission per edge. Moreover, we establish a number of theoretical guarantees for the performance of SBP, showing that it converges almost surely to the BP fixed point for any tree-structured graph, and for graphs with cycles satisfying a contractivity condition. In addition, for these graphical models, we provide non-asymptotic upper bounds on the convergence rate, showing that the $\ell_{\infty}$ norm of the error vector decays no slower than $O(1/\sqrt{t})$ with the number of iterations $t$ on trees and the mean square error decays as $O(1/t)$ for general graphs. These analysis show that SBP can provably yield reductions in computational and communication complexities for various classes of graphical models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑