arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2001.02178cs.CLphysics.data-an
收藏

标记语篇中的堆定律和堆函数:语言相关性的证据

Heaps' law and Heaps functions in tagged texts: Evidences of their linguistic relevance

Instituto de Física Enrique Gaviola(恩里克·加维奥拉物理研究所);Consejo Nacional de Investigaciones Científicas y Técnicas(国家科学技术研究委员会);Universidad Nacional de Córdoba(科尔多瓦国立大学);Centro Atómico Bariloche(巴里洛切原子中心);Instituto Balseiro(巴尔塞罗研究所);Comisión Nacional de Energía Atómica(国家原子能委员会);Universidad Nacional de Cuyo(库约国立大学)

Andrés Chacoma, Damián H. Zanette

摘要:

我们研究了词汇量和文本长度之间的关系,在一个语料库的$75$文学作品,由六个作家,区分三个语法类(或"标签“,即,{ \it名词},{ \it动词},和{ \it其他})的贡献,并分析每个标签的新词沿着每个单独的文本的渐进出现。虽然希普斯定律所规定的幂律关系是令人满意的总词汇量和文本长度,在每个文本中的新词的出现是在整体上很好地描述了平均随机洗牌的文本,它不服从幂律。然而,偏离这一平均值在统计上是显著的,并显示出整个语料库的系统趋势。具体来说,他们发现,沿每个文本的新词的出现主要是相对于随机洗牌的平均延迟。此外,不同的标签被证明增加系统的不同贡献,这种趋势,与{ \it动词}和{ \it其他}分别是更多和更少的平均趋势,而{ \it名词},而不是这个整体的平均值。这些统计系统性很可能表明存在存储在希普斯定律不同变体中的语言相关信息,这一特征仍需要广泛评估。

英文摘要:

We study the relationship between vocabulary size and text length in a corpus of $75$ literary works in English, authored by six writers, distinguishing between the contributions of three grammatical classes (or ``tags,'' namely, {\it nouns}, {\it verbs}, and {\it others}), and analyze the progressive appearance of new words of each tag along each individual text. While the power-law relation prescribed by Heaps' law is satisfactorily fulfilled by total vocabulary sizes and text lengths, the appearance of new words in each text is on the whole well described by the average of random shufflings of the text, which does not obey a power law. Deviations from this average, however, are statistically significant and show a systematic trend across the corpus. Specifically, they reveal that the appearance of new words along each text is predominantly retarded with respect to the average of random shufflings. Moreover, different tags are shown to add systematically distinct contributions to this tendency, with {\it verbs} and {\it others} being respectively more and less retarded than the mean trend, and {\it nouns} following instead this overall mean. These statistical systematicities are likely to point to the existence of linguistically relevant information stored in the different variants of Heaps' law, a feature that is still in need of extensive assessment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑