arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 0901.2349cs.CLcond-mat.dis-nnphysics.data-anphysics.soc-ph
收藏

超越词频:词的时间分布中的爆发、平静和缩放

Beyond word frequency: Bursts, lulls, and scaling in the temporal distributions of words

Northwestern University(西北大学)

Eduardo G. Altmann, Janet B. Pierrehumbert, Adilson E. Motter

摘要:

背景资料:齐普夫发现词频分布服从幂律,建立了生物和物理过程与语言之间的相似性,为人类交流的复杂系统观点奠定了基础。最近的研究也发现了事件连续发生的动力学中的缩放比例,这表明语言也可能有类似的发现。方法/主要发现:通过考虑频繁的话在USENET讨论组和在不同的数据库中的语言有不同程度的正式,在这里,我们表明,连续出现的同一个字显示突发偏离泊松过程之间的距离的分布,以及其特征在于拉伸指数(威布尔)缩放。这种偏离的程度在很大程度上取决于语义类型--衡量每个词的逻辑性--而不太依赖于频率。我们开发了这种行为的生成模型,完全决定了单词使用的动态。结论/意义:单词的重现模式可以通过重现时间的拉伸指数分布来很好地描述,这是一种无法从齐普夫定律中预测的经验缩放。由于文字的使用为人类的思想和活动提供了一个独特而精确的镜头,我们的发现也对人类集体动力的其他公开表现产生了影响。

英文摘要:

Background: Zipf's discovery that word frequency distributions obey a power law established parallels between biological and physical processes, and language, laying the groundwork for a complex systems perspective on human communication. More recent research has also identified scaling regularities in the dynamics underlying the successive occurrences of events, suggesting the possibility of similar findings for language as well. Methodology/Principal Findings: By considering frequent words in USENET discussion groups and in disparate databases where the language has different levels of formality, here we show that the distributions of distances between successive occurrences of the same word display bursty deviations from a Poisson process and are well characterized by a stretched exponential (Weibull) scaling. The extent of this deviation depends strongly on semantic type -- a measure of the logicality of each word -- and less strongly on frequency. We develop a generative model of this behavior that fully determines the dynamics of word usage. Conclusions/Significance: Recurrence patterns of words are well described by a stretched exponential distribution of recurrence times, an empirical scaling that cannot be anticipated from Zipf's law. Because the use of words provides a uniquely precise and powerful lens on human thought and activity, our findings also have implications for other overt manifestations of collective human dynamics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑