用于生成分级文本数据的马尔可夫链蒙特卡罗
Markov Chain Monte Carlo for generating ranked textual data
摘要:
本文面对的一个中心主题,在应用统计学和信息科学,这是随机结构的评估排名规模的法律文本分析。我们认为,在一个语料库中的话,他们排名的基础上,他们的频率降序。出发点是,在语言环境中产生的排名数据可以被看作是一个离散状态的实现马尔可夫链,其平稳分布的行为根据最佳拟合的秩大小法的离散化。所采用的方法学工具包是马尔可夫链蒙特卡罗,具体指的是梅特罗波利斯-黑斯廷斯算法。该理论框架被应用到美国总统演讲中的hapax legomena的排名大小分析。我们提供了大量的统计测试,从而使我们的方法论建议保持一致。为了追求我们的范围,我们也提供论据支持hapaxes是罕见的(“极端”)事件造成的内存少的过程。此外,我们认为,所考虑的样本有一个马尔可夫链的顺序一个随机结构。重要的是,我们讨论了该方法的多功能性,这被认为是适合于推导出类似的结果,为其他应用科学背景。
英文摘要:
This paper faces a central theme in applied statistics and information science, which is the assessment of the stochastic structure of rank-size laws in text analysis. We consider the words in a corpus by ranking them on the basis of their frequencies in descending order. The starting point is that the ranked data generated in linguistic contexts can be viewed as the realisations of a discrete states Markov chain, whose stationary distribution behaves according to a discretisation of the best fitted rank-size law. The employed methodological toolkit is Markov Chain Monte Carlo, specifically referring to the Metropolis-Hastings algorithm. The theoretical framework is applied to the rank-size analysis of the hapax legomena occurring in the speeches of the US Presidents. We offer a large number of statistical tests leading to the consistency of our methodological proposal. To pursue our scopes, we also offer arguments supporting that hapaxes are rare (``extreme") events resulting from memory-less-like processes. Moreover, we show that the considered sample has the stochastic structure of a Markov chain of order one. Importantly, we discuss the versatility of the method, which is considered suitable for deducing similar outcomes for other applied science contexts.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。