语言模型插值技术的连接与比较
Connecting and Comparing Language Model Interpolation Techniques
摘要:
在这项工作中,我们发现了两个语言模型插值技术,计数合并和贝叶斯插值之间的理论联系。我们比较了这些技术以及线性插值在三种情况下,每个组件模型具有丰富的训练数据。与以前的工作一致,我们表明,计数合并和贝叶斯插值优于线性插值。我们包括第一次(据我们所知)公布的计数合并和贝叶斯插值的比较,表明这两种技术的表现相似。最后,我们认为,其他考虑因素将使贝叶斯插值的首选方法在大多数情况下。
英文摘要:
In this work, we uncover a theoretical connection between two language model interpolation techniques, count merging and Bayesian interpolation. We compare these techniques as well as linear interpolation in three scenarios with abundant training data per component model. Consistent with prior work, we show that both count merging and Bayesian interpolation outperform linear interpolation. We include the first (to our knowledge) published comparison of count merging and Bayesian interpolation, showing that the two techniques perform similarly. Finally, we argue that other considerations will make Bayesian interpolation the preferred approach in most circumstances.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。