基于局部语法的编码再探
Local Grammar-Based Coding Revisited
AI总结：
本文针对最小局部语法编码领域，通过引入排序概率调和界、优化词汇量界、开发固定无限词汇普适码框架及分析经验排序有限词汇语法码，扩展了该领域基础并重申与幂律的关联。
摘要：
在最小局部语法编码设置中，输入字符串通过简单的逐符号编码表示为具有最小输出长度的语法。本文对该领域有四点贡献：首先，引入排序概率的简单调和界，其与齐普夫定律相关，简化了最小局部语法码的普适性证明；其次，优化词汇量的已知界，证明其与互信息和冗余度的部分幂律等价性，该界对关联齐普夫定律与大语言模型的神经缩放律具有意义；第三，开发固定无限词汇普适码框架，将普适编码重塑为匹配独立于经验数据的排序模式；最后，分析以经验排序列表为有限词汇的语法码，证明其同样普适。这些结果扩展了普适语法编码基础，重申了先前提出的与人类语言和语言模型幂律的关联。
英文摘要：
In the setting of minimal local grammar-based coding, the input string is represented as a grammar with the minimal output length defined via simple symbol-by-symbol encoding. This paper discusses four contributions to this field. First, we invoke a simple harmonic bound on ranked probabilities, which reminds Zipf's law and simplifies universality proofs for minimal local grammar-based codes. Second, we refine known bounds on the vocabulary size, showing its partial power-law equivalence with mutual information and redundancy. These bounds are relevant for linking Zipf's law with the neural scaling law for large language models. Third, we develop a framework for universal codes with fixed infinite vocabularies, recasting universal coding as matching ranked patterns that are independent of empirical data. Finally, we analyze grammar-based codes with finite vocabularies being empirical rank lists, proving that that such codes are also universal. These results extend foundations of universal grammar-based coding and reaffirm previously stated connections to power laws for human language and language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
