随机语言模型
Random Language Model
摘要:
许多复杂的生成系统使用语言来创建结构化对象。我们考虑一个模型的随机语言,定义加权上下文无关文法。随着语法权重分布的扩大,从一个随机的阶段,其中句子是无法区分的噪音,到一个有组织的阶段,其中携带非平凡的信息的过渡。这标志着语言中深层结构的出现,可以通过能量和熵之间的竞争来理解。
英文摘要:
Many complex generative systems use languages to create structured objects. We consider a model of random languages, defined by weighted context-free grammars. As the distribution of grammar weights broadens, a transition is found from a random phase, in which sentences are indistinguishable from noise, to an organized phase in which nontrivial information is carried. This marks the emergence of deep structure in the language, and can be understood by a competition between energy and entropy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。