arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1807.01855cs.CL
收藏

50种语言中的齐普夫定律:结构模式,语言解释和认知理据

Zipf's law in 50 languages: its structural pattern, linguistic interpretation, and cognitive motivation

Communication University of China(中国传媒大学);Anhui Jianzhu University(安徽建筑大学);Zhejiang University(浙江大学);Ningbo Institute of Technology, Zhejiang University(浙江大学宁波理工学院)

Shuiyuan Yu, Chunshan Xu, Haitao Liu

摘要:

齐普夫定律在许多与人类相关的领域都有发现,包括语言,其中一个词的频率一直被发现是其频率等级的幂律函数,称为齐普夫定律。然而,Zipf定律究竟是一个普遍规律还是一个统计学上的假象,人们对其形成的机制还知之甚少。为了回答这些问题,本研究对Zipf定律进行了大规模的跨语言调查。统计结果表明,50种语言中的齐普夫定律都具有三段结构模式,每段都有其独特的语言特性,下段总是向下弯曲,偏离理论预期。这一发现表明,这种偏离是自然语言中词频分布的一个基本和普遍的特征,而不是低频词的统计误差。基于双加工理论的计算机模拟得到了具有相同结构模式的Zipf定律,表明自然语言的Zipf定律是由共同的认知机制驱动的。这些结果表明,语言中的齐普夫定律是由控制人类言语行为的双重加工等认知机制驱动的。

英文摘要:

Zipf's law has been found in many human-related fields, including language, where the frequency of a word is persistently found as a power law function of its frequency rank, known as Zipf's law. However, there is much dispute whether it is a universal law or a statistical artifact, and little is known about what mechanisms may have shaped it. To answer these questions, this study conducted a large scale cross language investigation into Zipf's law. The statistical results show that Zipf's laws in 50 languages all share a 3-segment structural pattern, with each segment demonstrating distinctive linguistic properties and the lower segment invariably bending downwards to deviate from theoretical expectation. This finding indicates that this deviation is a fundamental and universal feature of word frequency distributions in natural languages, not the statistical error of low frequency words. A computer simulation based on the dual-process theory yields Zipf's law with the same structural pattern, suggesting that Zipf's law of natural languages are motivated by common cognitive mechanisms. These results show that Zipf's law in languages is motivated by cognitive mechanisms like dual-processing that govern human verbal behaviors.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑