arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 1710.07960cs.CL

How big is big enough? Unsupervised word sense disambiguation using a very large corpus

Institute of Computer Science, Polish Academy of Sciences(波兰科学院计算机科学研究所);National Centre for Text Mining, University of Manchester(曼彻斯特大学国家文本挖掘中心)

Piotr Przybyła

英文摘要:

In this paper, the problem of disambiguating a target word for Polish is approached by searching for related words with known meaning. These relatives are used to build a training corpus from unannotated text. This technique is improved by proposing new rich sources of replacements that substitute the traditional requirement of monosemy with heuristics based on wordnet relations. The naïve Bayesian classifier has been modified to account for an unknown distribution of senses. A corpus of 600 million web documents (594 billion tokens), gathered by the NEKST search engine allows us to assess the relationship between training set size and disambiguation accuracy. The classifier is evaluated using both a wordnet baseline and a corpus with 17,314 manually annotated occurrences of 54 ambiguous words.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑