arXivDaily arXiv每日学术速递 周一至周五更新
arXiv cs/0305041cs.CL
收藏

基于退避格的语言模型因子分解

Factorization of Language Models through Backing-Off Lattices

Wei Wang

摘要:

统计语言模型的因子分解是将最具鉴别力的模型分解为因子化模型,并通过组合它们来确定一个新的模型,以提供更好的估计。以往的工作主要集中在序列事件的因子分解模型上,每个模型只允许一种因子分解方式。为了实现并行因子分解,这允许模型事件同时以多种方式被解析,我们提出了一个通用框架,其中我们采用退避格来反映并行因子分解并定义模型被解析为因子模型的路径,我们使用混合模型来组合格中的并行路径,推广Katz的backing-off方法,将遍历整个格得到的混合模型进行积分。基于这个框架,我们制定了两种类型的模型因子分解,用于自然语言建模。

英文摘要:

Factorization of statistical language models is the task that we resolve the most discriminative model into factored models and determine a new model by combining them so as to provide better estimate. Most of previous works mainly focus on factorizing models of sequential events, each of which allows only one factorization manner. To enable parallel factorization, which allows a model event to be resolved in more than one ways at the same time, we propose a general framework, where we adopt a backing-off lattice to reflect parallel factorizations and to define the paths along which a model is resolved into factored models, we use a mixture model to combine parallel paths in the lattice, and generalize Katz's backing-off method to integrate all the mixture models got by traversing the entire lattice. Based on this framework, we formulate two types of model factorizations that are used in natural language modeling.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑