arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.02772physics.soc-phcs.CLnlin.AOstat.AP
收藏

自然语言的复杂系统方法

Complex systems approach to natural language

Institute of Nuclear Physics, Polish Academy of Sciences(波兰科学院核物理研究所);Cracow University of Technology(克拉科夫工业大学)

Tomasz Stanisz, Stanisław Drożdż, Jarosław Kwapień

AI总结:

本综述从复杂性科学视角总结了自然语言研究的三大方法论趋势,涵盖词频标度、时间序列相关性与网络形式体系,强调标点符号在恢复语言统计特性和信息承载能力中的关键作用。

摘要:

本综述总结了从复杂性科学视角研究自然语言所使用的主要方法论概念,并记录了这些方法在识别书面语言表征中普遍特征与系统特定特征方面的适用性。文中涵盖了定量语言学中三个主要的与复杂性相关的研究趋势。第一部分探讨了文本中的词频问题,并表明考虑标点符号能够恢复标度特性,而对于最频繁的词,通常能观察到其违反 Zipf 定律的现象。第二部分介绍了受时间序列分析启发的方法,用于研究书面文本中各类相关性。相关时间序列是基于将文本划分为句子或连续标点符号之间的短语来生成的。结果表明,这些序列展现出复杂系统生成的信号中常见的特征,如长程相关性或(多重)分形结构。此外,标点符号之间的距离似乎符合 Weibull 分布的离散变体。第三部分回顾了网络形式体系在自然语言中的应用,特别是在所谓的词邻接网络背景下。表征此类网络拓扑的参数可用于文本分类,例如从文体测量学角度。网络方法也可用于表示词联想的组织结构。词联想网络的结构与随机网络中观察到的结构显著不同,揭示了语言的固有属性。最后,标点符号似乎不仅对语言的信息承载能力有显著影响,而且对其关键统计特性也有显著影响,因此建议将标点符号与词同等对待。

英文摘要:

The review summarizes the main methodological concepts used in studying natural language from the perspective of complexity science and documents their applicability in identifying both universal and system-specific features of language in its written representation. Three main complexity-related research trends in quantitative linguistics are covered. The first part addresses the issue of word frequencies in texts and demonstrates that taking punctuation into consideration restores scaling whose violation in the Zipf's law is often observed for the most frequent words. The second part introduces methods inspired by time series analysis, used in studying various kinds of correlations in written texts. The related time series are generated on the basis of text partition into sentences or into phrases between consecutive punctuation marks. It turns out that these series develop features often found in signals generated by complex systems, like long-range correlations or (multi)fractal structures. Moreover, it appears that the distances between punctuation marks comply with the discrete variant of the Weibull distribution. In the third part, the application of the network formalism to natural language is reviewed, particularly in the context of the so-called word-adjacency networks. Parameters characterizing topology of such networks can be used for classification of texts, for example, from a stylometric perspective. Network approach can also be applied to represent the organization of word associations. Structure of word-association networks turns out to be significantly different from that observed in random networks, revealing genuine properties of language. Finally, punctuation seems to have a significant impact not only on the language's information-carrying ability but also on its key statistical properties, hence it is recommended to consider punctuation marks on a par with words.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑