arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1105.1853stat.MLcs.AI
收藏

高斯图模型中的反馈信息传递推理

Feedback Message Passing for Inference in Gaussian Graphical Models

Massachusetts Institute of Technology（麻省理工学院）；University of California, Irvine（加州大学欧文分校）

Ying Liu, Venkat Chandrasekaran, Animashree Anandkumar, Alan S. Willsky

备注：
30 pages
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

摘要：

虽然循环置信传播（LBP）在一些具有循环的高斯图形模型中的推理表现相当不错，但它的性能对许多其他模型并不令人满意。特别地，对于某些模型，LBP不收敛，并且通常当它收敛时，计算的方差是不正确的（除了无环图，其置信传播（BP）是非迭代和精确的）。在本文中，我们提出了{\em反馈消息传递}（FMP），一个消息传递算法，利用一个特殊的顶点集（称为{\em反馈顶点集}或{\em FVS}），其删除结果在一个无圈图。在FMP中，标准BP被多次使用在不包括FVS的无环子图上，而一个特殊的消息传递方案被用于FVS中的节点。精确推理的计算复杂度为$O(k^2n)$，其中$k$为反馈节点数，$n$为节点总数。当FVS的尺寸非常大时，FMP是难处理的。因此，我们提出{\em approximate FMP}，其中使用伪FVS而不是FVS，并且其中通过去除伪FVS而获得的非无圈图中的推断使用LBP近似地进行。我们表明，当近似FMP收敛时，它会产生确切的手段和伪FVS和确切的手段在整个图的其余部分的方差。我们还提供了近似FMP的收敛性和精度的理论结果。特别是，我们证明方差计算的误差界。基于这些理论结果，我们设计了有效的算法来选择一个有界大小的伪FVS。伪FVS的选择允许我们明确地在效率和准确性之间进行权衡。实验结果表明，使用大小不大于$\log(n)$的伪FVS，该过程比LBP在整个图上更频繁、更快速地收敛，并且提供更准确的结果。

英文摘要：

While loopy belief propagation (LBP) performs reasonably well for inference in some Gaussian graphical models with cycles, its performance is unsatisfactory for many others. In particular for some models LBP does not converge, and in general when it does converge, the computed variances are incorrect (except for cycle-free graphs for which belief propagation (BP) is non-iterative and exact). In this paper we propose {\em feedback message passing} (FMP), a message-passing algorithm that makes use of a special set of vertices (called a {\em feedback vertex set} or {\em FVS}) whose removal results in a cycle-free graph. In FMP, standard BP is employed several times on the cycle-free subgraph excluding the FVS while a special message-passing scheme is used for the nodes in the FVS. The computational complexity of exact inference is $O(k^2n)$, where $k$ is the number of feedback nodes, and $n$ is the total number of nodes. When the size of the FVS is very large, FMP is intractable. Hence we propose {\em approximate FMP}, where a pseudo-FVS is used instead of an FVS, and where inference in the non-cycle-free graph obtained by removing the pseudo-FVS is carried out approximately using LBP. We show that, when approximate FMP converges, it yields exact means and variances on the pseudo-FVS and exact means throughout the remainder of the graph. We also provide theoretical results on the convergence and accuracy of approximate FMP. In particular, we prove error bounds on variance computation. Based on these theoretical results, we design efficient algorithms to select a pseudo-FVS of bounded size. The choice of the pseudo-FVS allows us to explicitly trade off between efficiency and accuracy. Experimental results show that using a pseudo-FVS of size no larger than $\log(n)$, this procedure converges much more often, more quickly, and provides more accurate results than LBP on the entire graph.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑