探索未知文本的统计特性：应用于伏尼契手稿
Probing the statistical properties of unknown texts: application to the Voynich Manuscript
摘要：
虽然使用统计物理学方法来分析大型语料库已经有助于揭示文本中的许多模式，但还没有进行全面的调查，调查不同语言和文本之间的统计测量的属性。在这项研究中，我们提出了一个框架，旨在确定文本是否与自然语言兼容，以及哪些语言最接近它，而不需要了解单词的含义。该方法是基于三种类型的统计测量，即获得从一阶统计的文字属性的文本，从复杂的网络表示文本的拓扑结构，并从连续性的概念，其中文本被视为一个时间序列。比较实验用15种不同语言的《新约》和英语和葡萄牙语的不同书籍进行，以量化不同测量对语言和书中讲述的故事的依赖性。发现在区分真实文本与其混洗版本方面具有信息性的指标包括词的重复性、程度和选择性。作为一个例子，我们分析了一个未破译的中世纪手稿被称为伏尼契手稿。我们表明，它主要是兼容的自然语言和不兼容的随机文本。我们还获得候选人的关键词的伏尼契手稿，这可能是有帮助的努力破译it.Because我们能够确定统计测量，更依赖于语法比语义，该框架也可用于文本分析语言相关的应用程序。
英文摘要：
While the use of statistical physics methods to analyze large corpora has been useful to unveil many patterns in texts, no comprehensive investigation has been performed investigating the properties of statistical measurements across different languages and texts. In this study we propose a framework that aims at determining if a text is compatible with a natural language and which languages are closest to it, without any knowledge of the meaning of the words. The approach is based on three types of statistical measurements, i.e. obtained from first-order statistics of word properties in a text, from the topology of complex networks representing text, and from intermittency concepts where text is treated as a time series. Comparative experiments were performed with the New Testament in 15 different languages and with distinct books in English and Portuguese in order to quantify the dependency of the different measurements on the language and on the story being told in the book. The metrics found to be informative in distinguishing real texts from their shuffled versions include assortativity, degree and selectivity of words. As an illustration, we analyze an undeciphered medieval manuscript known as the Voynich Manuscript. We show that it is mostly compatible with natural languages and incompatible with random texts. We also obtain candidates for key-words of the Voynich Manuscript which could be helpful in the effort of deciphering it. Because we were able to identify statistical measurements that are more dependent on the syntax than on the semantics, the framework may also serve for text analysis in language-dependent applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
