The dependence with text length of the statistical properties of word occurrences has long been considered a severe limitation quantitative linguistics. We propose a simple scaling form for the distribution of absolute word frequencies which uncovers the robustness of this distribution as text grows. In this way, the shape of the distribution is always the same and it is only a scale parameter which increases linearly with text length. By analyzing very long novels we show that this behavior holds both for raw, unlemmatized texts and for lemmatized texts. For the latter case, the word-frequency distribution is well fit by a double power law, maintaining the Zipf's exponent value $γ\simeq 2$ for large frequencies but yielding a smaller exponent in the low frequency regime. The growth of the distribution with text length allows us to estimate the size of the vocabulary at each step and to propose an alternative to Heaps' law, which turns out to be intimately connected to Zipf's law, thanks to the scaling behavior.
Complex natural and technological systems can be considered, on a coarse-grained level, as assemblies of elementary components: for example, genomes as sets of genes, or texts as sets of words. On one hand, the joint occurrence of components emerges from architectural and specific constraints in such systems. On the other hand, general regularities may unify different systems, such as the broadly studied Zipf and Heaps laws, respectively concerning the distribution of component frequencies and their number as a function of system size. Dependency structures (i.e., directed networks encoding th…
Zipf's, Heaps' and Taylor's laws are ubiquitous in many different systems where innovation processes are at play. Together, they represent a compelling set of stylized facts regarding the overall statistics, the innovation rate and the scaling of fluctuations for systems as diverse as written texts and cities, ecological systems and stock markets. Many modeling schemes have been proposed in literature to explain those laws, but only recently a modeling framework has been introduced that accounts for the emergence of those laws without deducing the emergence of one of the laws from the others o…
The dynamics of discovery and the Heaps-Zipf relationship
CNRS, UMR Géographie-cités & LabCom MIXTAPES（法国国家科学研究中心，地理城市联合研究单位及MIXTAPES实验室合作委员会）；Deezer research & LabCom MIXTAPES（Deezer 研究部门及MIXTAPES实验室合作委员会）；Université Paris-Saclay, CNRS, CEA, Institut de Physique Théorique（巴黎萨克雷大学，法国国家科学研究中心，法国原子能和替代能源委员会，理论物理研究所）；Centre d’Analyse et de Mathématique Sociales (CNRS/EHESS)（社会分析与数学中心（法国国家科学研究中心/高等社会科学学院））
Zipf's law is a hallmark of several complex systems with a modular structure, such as books composed by words or genomes composed by genes. In these component systems, Zipf's law describes the empirical power law distribution of component frequencies. Stochastic processes based on a sample-space-reducing (SSR) mechanism, in which the number of accessible states reduces as the system evolves, have been recently proposed as a simple explanation for the ubiquitous emergence of this law. However, many complex component systems are characterized by other statistical patterns beyond Zipf's law, such…
Universalities in the Avalanche Dynamics of Novelties and Non-Novelties and some Notes on the Heaps law
CREF - Centro Ricerche Enrico Fermi（恩里科·费米研究中心）；Sapienza University of Rome（罗马大学）；CNR - Institute of Complex Systems（复杂系统研究所）；Complexity Science Hub Vienna（维也纳复杂性科学中心）
The rank-size distribution of cities follows Zipf's law, and the Zipf scaling exponent often tends to a constant 1. This seems to be a general rule. However, a recent numerical experiment shows that there exists a contradiction between the Zipf exponent 1 and high urbanization level in a large population country. In this paper, mathematical modeling, computational analysis, and the method of proof by contradiction are employed to reveal the numerical relationships between urbanization level and Zipf scaling exponent. The main findings are as follows. (1) If Zipf scaling exponent equals 1, the …
We propose hypotheses describing the empirical finding of an association between the exponents of urban GDP scaling and Zipf's law for cities. These hypotheses represent various combinations of directional or reciprocal causal links between the two phenomena and include inter- and intra-city processes. Future theories and models can be motivated with and categorized according to these hypotheses. This paper intends to stimulate the discussion around the processes behind these phenomena and pave the way to a Unified Urban Theory.
Beyond Zipf's law: Modeling the structure of human language
IFISC (CSIC-UIB) Instituto de Física Interdisciplinar y Sistemas Complejos（IFISC（西班牙国家研究委员会-巴利阿里群岛大学）跨学科物理与复杂系统研究所）；Indiana University（印第安纳大学）；ISI Foundation（ISI基金会）
An important body of quantitative linguistics is constituted by a series of statistical laws about language usage. Despite the importance of these linguistic laws, some of them are poorly formulated, and, more importantly, there is no unified framework that encompasses all them. This paper presents a new perspective to establish a connection between different statistical linguistic laws. Characterizing each word type by two random variables, length (in number of characters) and absolute frequency, we show that the corresponding bivariate joint probability distribution shows a rich and precise …
Urban scaling and Zipf's law are two fundamental paradigms for the science of cities. These laws have mostly been investigated independently and are often perceived as disassociated matters. Here we present a large scale investigation about the connection between these two laws using population and GDP data from almost five thousand consistently-defined cities in 96 countries. We empirically demonstrate that both laws are tied to each other and derive an expression relating the urban scaling and Zipf exponents. This expression captures the average tendency of the empirical relation between bot…
The word-frequency distribution provides the fundamental building blocks that generate discourse in language. It is well known, from empirical evidence, that the word-frequency distribution of almost any text is described by Zipf's law, at least approximately. Following Stephens and Bialek [Phys. Rev. E 81, 066119, 2010], we interpret the frequency of any word as arising from the interaction potential between its constituent letters. Indeed, Jaynes' maximum-entropy principle, with the constrains given by every empirical two-letter marginal distribution, leads to a Boltzmann distribution for wo…
Eötvös Loránd University（厄特沃什·罗兰大学）；Senseable City Laboratory, MIT（麻省理工学院感知城市实验室）；Singapore-MIT Alliance for Research and Technology（新加坡-麻省理工学院研究与技术联盟）
Scaling properties of language are a useful tool for understanding generative processes in texts. We investigate the scaling relations in citywise Twitter corpora coming from the Metropolitan and Micropolitan Statistical Areas of the United States. We observe a slightly superlinear urban scaling with the city population for the total volume of the tweets and words created in a city. We then find that a certain core vocabulary follows the scaling relationship of that of the bulk text, but most words are sensitive to city size, exhibiting a super- or a sublinear urban scaling. For both regimes w…
Understanding the innovation process, that is the underlying mechanisms through which novelties emerge, diffuse and trigger further novelties is undoubtedly of fundamental importance in many areas (biology, linguistics, social science and others). The models introduced so far satisfy the Heaps' law, regarding the rate at which novelties appear, and the Zipf's law, that states a power law behavior for the frequency distribution of the elements. However, there are empirical cases far from showing a pure power law behavior and such a deviation is present for elements with high frequencies. We exp…
Zipf's law states that the frequency of an observation with a given value is inversely proportional to the square of that value; Taylor's law, instead, describes the scaling between fluctuations in the size of a population and its mean. Empirical evidence of the validity of these laws has been found in many and diverse domains. Despite the numerous models proposed to explain the presence of Zipf's law, there is no consensus on how it originates from a microscopic process of individuals dynamics without fine tuning. Here we show that Zipf's law and Taylor's law can emerge from a general class o…
The rank-size plots of a large number of different physical and socio-economic systems are usually said to follow Zipf's law, but a unique framework for the comprehension of this ubiquitous scaling law is still lacking. Here we show that a dynamical approach is crucial: during their evolution, some systems are attracted towards Zipf's law, while others presents Zipf's law only temporarily and, therefore, spuriously. A truly Zipfian dynamics is characterized by a dynamical constraint, or coherence, among the parameters of the generating PDF, and the number of elements in the system. A clear-cut…
In this work we analyze statistical properties of 91 relatively small texts in 7 different languages (Spanish, English, French, German, Turkish, Russian, Icelandic) as well as texts with randomly inserted spaces. Despite the size (around 11260 different words), the well known universal statistical laws -- namely Zipf and Herdan-Heap's laws -- are confirmed, and are in close agreement with results obtained elsewhere. We also construct a word co-occurrence network of each text. While the degree distribution is again universal, we note that the distribution of Clustering Coefficients, which depen…
Departamento de Física, Universidade Federal de Pernambuco（物理系，佩纳伯库联邦大学）；Instituto Federal de Educação, Ciência e Tecnologia de Pernambuco（巴雷里奥斯佩纳伯库教育科技研究所）
Some authors have recently argued that a finite-size scaling law for the text-length dependence of word-frequency distributions cannot be conceptually valid. Here we give solid quantitative evidence for the validity of such scaling law, both using careful statistical tests and analytical arguments based on the generalized central-limit theorem applied to the moments of the distribution (and obtaining a novel derivation of Heaps' law as a by-product). We also find that the picture of word-frequency distributions with power-law exponents that decrease with text length [Yan and Minnhagen, Physica…
Research institutions provide the infrastructure for scientific discovery, yet their role in the production of knowledge is not well characterized. To address this gap, we analyze interactions of researchers within and between institutions from millions of scientific papers. Our analysis reveals that the number of collaborations scales superlinearly with institution size, though at different rates (heterogeneous densification). We also find that the number of institutions scales with the number of researchers as a power law (Heaps' law) and institution sizes approximate Zipf's law. These patte…
Universidad Mayor（马约尔大学）；Universidad Adolfo Ibáñez（阿道夫·伊瓦涅斯大学）
The Zipf's law establishes that if the words of a (large) text are ordered by decreasing frequency, the frequency versus the rank decreases as a power law with exponent close to $-1$. Previous work has stressed that this pattern arises from a conflict of interests of the participants of communication. The challenge here is to define a computational multi-agent language game, mainly based on a parameter that measures the relative participant's interests. Numerical simulations suggest that at critical values of the parameter a human-like vocabulary, exhibiting scaling properties, seems to appear…
Using population data of high spatial resolution for a region in the south of Europe, we define cities by aggregating individuals to form connected clusters. The resulting cluster-population distributions show a smooth decreasing behavior covering six orders of magnitude. We perform a detailed study of the distributions, using state-of-the-art statistical tools. By means of scaling analysis we rule out the existence of a power-law regime in the low-population range. The logarithmic-coefficient-of-variation test allows us to establish that the power-law tail for high population, characteristic …
