arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1805.01460cs.CLphysics.soc-ph
收藏

书面语篇中句子长度测量的鲁棒性

Robustness of sentence length measures in written texts

Universidade Estadual de Maringá(马林加州立大学)

Denner S. Vieira, Sergio Picoli, Renio S. Mendes

摘要:

在过去的几十年里,书面语篇中隐藏的结构模式一直是大量研究的主题。特别是,将文本映射到句子长度的时间序列是研究文本结构的自然方法。通常,句子长度已经通过使用基于单词数量和字符数量的度量来量化,但其他变化也是可能的。为了量化不同句子长度度量的鲁棒性,我们分析了包含约500本英语书籍的数据库。对于每本书,我们提取了六个不同的句子长度度量,包括单词数量和字符数量(考虑到词形还原和停止词删除)。我们比较了这六个措施,为每本书使用i)皮尔逊系数调查线性相关性; ii)Kolmogorov-Smirnov测试比较分布;和iii)去趋势波动分析(DFA)量化自相关性。我们发现,所有六个指标都表现出非常相似的行为,这表明句子长度是与文本结构相关的鲁棒指标。

英文摘要:

Hidden structural patterns in written texts have been subject of considerable research in the last decades. In particular, mapping a text into a time series of sentence lengths is a natural way to investigate text structure. Typically, sentence length has been quantified by using measures based on the number of words and the number of characters, but other variations are possible. To quantify the robustness of different sentence length measures, we analyzed a database containing about five hundred books in English. For each book, we extracted six distinct measures of sentence length, including number of words and number of characters (taking into account lemmatization and stop words removal). We compared these six measures for each book by using i) Pearson's coefficient to investigate linear correlations; ii) Kolmogorov--Smirnov test to compare distributions; and iii) detrended fluctuation analysis (DFA) to quantify auto-correlations. We have found that all six measures exhibit very similar behavior, suggesting that sentence length is a robust measure related to text structure.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑