人类语言中的尺度律
Scaling Laws in Human Language
摘要：
Zipf的词频定律在英语、法语、西班牙语、意大利语等语言中都可以观察到，但它并不适用于中文、日语或韩语字符。本文提出了一个写作过程模型来解释上述差异，该模型考虑了词汇量有限的影响。实验、模拟和解析解吻合较好。结果表明，频率分布服从指数为1的幂律，在指数为1处，相应的Zipf指数发散。实际上，在Zipf图中，分布服从指数形式。与堆定律不同的是，不同词的数量随着文本长度的增加而增加，并经历了三个阶段：一开始是线性增长，然后变成数学形式，最后达到饱和。这一工作完善了以前对语言系统中Zipf定律和Heaps定律的理解。
英文摘要：
Zipf's law on word frequency is observed in English, French, Spanish, Italian, and so on, yet it does not hold for Chinese, Japanese or Korean characters. A model for writing process is proposed to explain the above difference, which takes into account the effects of finite vocabulary size. Experiments, simulations and analytical solution agree well with each other. The results show that the frequency distribution follows a power law with exponent being equal to 1, at which the corresponding Zipf's exponent diverges. Actually, the distribution obeys exponential form in the Zipf's plot. Deviating from the Heaps' law, the number of distinct words grows with the text length in three stages: It grows linearly in the beginning, then turns to a logarithmical form, and eventually saturates. This work refines previous understanding about Zipf's law and Heaps' law in language systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
