文本光谱分析：语言与话语边界的自动检测
Analyse spectrale des textes: détection automatique des frontières de langue et de discours
摘要：
我们提出了一个理论框架，在此框架内，可以推断的基础上收集的统计信息，语料库的词汇信息。可以根据词汇表中单词的类别以及它们在特定语言中的句法属性进行推断。基于相同的统计数据，可以构建组合相似性矩阵（二元语法转换矩阵）或聚合相似性矩阵（任何一对单词共享共同上下文的概率）。当相对于它们的句法相似性聚类时，单词倾向于分组为子语言词汇，并且当相对于它们的聚合相似性聚类时，分组为句法或语义类。实验已经探索了这两种可能性中的第一种。他们的结果被解释的语料库的生成过程（S）的马尔可夫链建模的框架：我们表明，转换矩阵的频谱分析的结果可以被解释为集群内的单词的概率分布。这种方法产生了一个软聚类的词汇到子语言，这有助于异构语料库的生成。作为一个应用程序，我们展示了如何多语言文本可以在视觉上分割成语言同质段。我们的方法是特别有用的情况下，发生在语料库中的混合相关的语言。
英文摘要：
We propose a theoretical framework within which information on the vocabulary of a given corpus can be inferred on the basis of statistical information gathered on that corpus. Inferences can be made on the categories of the words in the vocabulary, and on their syntactical properties within particular languages. Based on the same statistical data, it is possible to build matrices of syntagmatic similarity (bigram transition matrices) or paradigmatic similarity (probability for any pair of words to share common contexts). When clustered with respect to their syntagmatic similarity, words tend to group into sublanguage vocabularies, and when clustered with respect to their paradigmatic similarity, into syntactic or semantic classes. Experiments have explored the first of these two possibilities. Their results are interpreted in the frame of a Markov chain modelling of the corpus' generative processe(s): we show that the results of a spectral analysis of the transition matrix can be interpreted as probability distributions of words within clusters. This method yields a soft clustering of the vocabulary into sublanguages which contribute to the generation of heterogeneous corpora. As an application, we show how multilingual texts can be visually segmented into linguistically homogeneous segments. Our method is specifically useful in the case of related languages which happened to be mixed in corpora.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
