arXiv 1509.089922015-09cs.LGstat.ML

基于快速混合参数集的任意树宽最大似然学习

Maximum Likelihood Learning With Arbitrary Treewidth via Fast-Mixing Parameter Sets

NICTA（国家信息与通信技术研究院）；Australian National University（澳大利亚国立大学）

Justin Domke

备注：: Advances in Neural Information Processing Systems 2015

摘要：

在高树宽无向图模型中，推理通常是难以处理的，这使得最大似然学习成为一个挑战。克服这一点的一种方法是将参数限制为易于处理的集合，最典型的是树结构参数的集合。本文探讨了一个易处理的集合，即一组“快速混合参数”，马尔可夫链蒙特卡罗（MCMC）推断可以保证快速收敛到平稳分布的替代概念。虽然在实践中使用从MCMC获得的样本来近似似然梯度是常见的，但是这样的程序缺乏理论保证。本文证明了对于任何具有有界充分统计量的指数族（不仅仅是图模型），当参数被约束在一个快速混合集上时，用采样近似梯度的梯度下降将以高概率逼近集内的最大似然解.当未正则化时，要找到对数似然的ε-精度解，需要1/1/2的总工作量，忽略对数因子。当脊正则化时，强凸性允许解在参数距离上是ε-精确的，而努力是1/ε的二次。这两种方法都提供了一个完全多项式时间的随机近似方案。

英文摘要：

Inference is typically intractable in high-treewidth undirected graphical models, making maximum likelihood learning a challenge. One way to overcome this is to restrict parameters to a tractable set, most typically the set of tree-structured parameters. This paper explores an alternative notion of a tractable set, namely a set of "fast-mixing parameters" where Markov chain Monte Carlo (MCMC) inference can be guaranteed to quickly converge to the stationary distribution. While it is common in practice to approximate the likelihood gradient using samples obtained from MCMC, such procedures lack theoretical guarantees. This paper proves that for any exponential family with bounded sufficient statistics, (not just graphical models) when parameters are constrained to a fast-mixing set, gradient descent with gradients approximated by sampling will approximate the maximum likelihood solution inside the set with high-probability. When unregularized, to find a solution epsilon-accurate in log-likelihood requires a total amount of effort cubic in 1/epsilon, disregarding logarithmic factors. When ridge-regularized, strong convexity allows a solution epsilon-accurate in parameter distance with effort quadratic in 1/epsilon. Both of these provide of a fully-polynomial time randomized approximation scheme.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文