斯里兰卡十年来的僧伽罗语语料库和停用词Facebook
Sinhala Language Corpora and Stopwords from a Decade of Sri Lankan Facebook
摘要:
本文介绍了LIRNEasia的数据,分析和政策团队的语言努力中的两个口语僧伽罗语语料库,以及算法派生的停用词列表。两个语料库中较大的一个跨越2010年至2020年,包含533个斯里兰卡Facebook页面发布的28,825,820至29,549,672个单词的多语言文本,包括政治,媒体,名人和其他类别;较小的语料库只有5,402,76个单词的僧伽罗语文本。这两个语料库都有标记,用于标记它们的创建日期、来源页面和内容类型。
英文摘要:
This paper presents two colloquial Sinhala language corpora from the language efforts of the Data, Analysis and Policy team of LIRNEasia, as well as a list of algorithmically derived stopwords. The larger of the two corpora spans 2010 to 2020 and contains 28,825,820 to 29,549,672 words of multilingual text posted by 533 Sri Lankan Facebook pages, including politics, media, celebrities, and other categories; the smaller corpus amounts to 5,402,76 words of only Sinhala text extracted from the larger. Both corpora have markers for their date of creation, page of origin, and content type.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。