马尔可夫树结构极大似然学习的大偏差分析
A Large-Deviation Analysis of the Maximum-Likelihood Learning of Markov Tree Structures
摘要:
研究了离散树型分布的极大似然估计问题。Chow和Liu建立了ML-估计减少到使用经验互信息量作为边权重的最大权重生成树的构造。使用大偏差理论,我们分析了与事件的错误概率相关联的指数,该事件的马尔可夫树结构的ML-估计不同于真正的树结构,给定一组独立抽取的样本。通过利用ML估计的输出是一棵树的事实,我们建立了误差指数等于单个显性交叉事件的指数衰减率。我们证明,在这个占主导地位的交叉事件中,一个非邻居节点对取代了真正的边缘的分布是沿着路径的边缘在真正的树图连接的节点在非邻居对。利用欧几里得信息论的思想,我们分析了在非常嘈杂的学习机制中ML估计的情况,并表明误差指数可以近似为一个比率,该比率被解释为学习树分布的信噪比(SNR)。我们通过数值实验表明,在这种制度下,我们的SNR近似是准确的。
英文摘要:
The problem of maximum-likelihood (ML) estimation of discrete tree-structured distributions is considered. Chow and Liu established that ML-estimation reduces to the construction of a maximum-weight spanning tree using the empirical mutual information quantities as the edge weights. Using the theory of large-deviations, we analyze the exponent associated with the error probability of the event that the ML-estimate of the Markov tree structure differs from the true tree structure, given a set of independently drawn samples. By exploiting the fact that the output of ML-estimation is a tree, we establish that the error exponent is equal to the exponential rate of decay of a single dominant crossover event. We prove that in this dominant crossover event, a non-neighbor node pair replaces a true edge of the distribution that is along the path of edges in the true tree graph connecting the nodes in the non-neighbor pair. Using ideas from Euclidean information theory, we then analyze the scenario of ML-estimation in the very noisy learning regime and show that the error exponent can be approximated as a ratio, which is interpreted as the signal-to-noise ratio (SNR) for learning tree distributions. We show via numerical experiments that in this regime, our SNR approximation is accurate.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。