对MMI准则的逼近及其对基于格的MMI的影响
Approximations to the MMI criterion and their effect on lattice-based MMI
摘要:
最大互信息(MMI)是一种用于隐马尔可夫模型(HMM)参数估计的模型选择准则,其在二十多年前被开发为用于基于HMM的语音识别的最大似然准则的区别性替代。在语音识别文献中已经表明,使用当前MMI范例(基于格的MMI)的参数估计始终优于最大似然估计,但这是以不期望的收敛特性为代价的。特别是,识别性能是敏感的迭代MMI估计算法,扩展鲍姆-韦尔奇,执行的次数。事实上,太多的迭代扩展Baum-Welch将导致性能下降,尽管事实上,MMI标准在每次迭代的改进。这一现象与最大似然估计的类似行为不同--至少对于语音识别中使用的HSP 70来说是这样--并且它以前被归因于“过拟合”。在本文中,我们提出了一个基于格的MMI的分析表明,首先,基于格的MMI的渐近行为比以前理解的要差得多,即它似乎根本不收敛,其次,这不是由于“过拟合”。相反,我们证明了“过拟合”现象是标准方法的结果,加剧了两个关键的近似在基于格的MMI机器的不良行为。我们还表明,如果我们修改的标准方法,以提高这些近似的有效性,那么基于格的MMI的收敛特性变得良性,而不牺牲提高识别精度。
英文摘要:
Maximum mutual information (MMI) is a model selection criterion used for hidden Markov model (HMM) parameter estimation that was developed more than twenty years ago as a discriminative alternative to the maximum likelihood criterion for HMM-based speech recognition. It has been shown in the speech recognition literature that parameter estimation using the current MMI paradigm, lattice-based MMI, consistently outperforms maximum likelihood estimation, but this is at the expense of undesirable convergence properties. In particular, recognition performance is sensitive to the number of times that the iterative MMI estimation algorithm, extended Baum-Welch, is performed. In fact, too many iterations of extended Baum-Welch will lead to degraded performance, despite the fact that the MMI criterion improves at each iteration. This phenomenon is at variance with the analogous behavior of maximum likelihood estimation -- at least for the HMMs used in speech recognition -- and it has previously been attributed to `over fitting'. In this paper, we present an analysis of lattice-based MMI that demonstrates, first of all, that the asymptotic behavior of lattice-based MMI is much worse than was previously understood, i.e. it does not appear to converge at all, and, second of all, that this is not due to `over fitting'. Instead, we demonstrate that the `over fitting' phenomenon is the result of standard methodology that exacerbates the poor behavior of two key approximations in the lattice-based MMI machinery. We also demonstrate that if we modify the standard methodology to improve the validity of these approximations, then the convergence properties of lattice-based MMI become benign without sacrificing improvements to recognition accuracy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。