arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.05381cs.CLcs.AIcs.LG

CCAE:基于中文的亚洲英语语料库

CCAE: A Corpus of Chinese-based Asian Englishes

University of Science and Technology Beijing(北京科技大学)

Yang Liu, Melissa Xiaohui Qin, Long Wang, Chao Huang

AI总结:

本研究构建了包含6种基于中文的亚洲英语变体、规模达3.4亿token的CCAE语料库,填补了中式英语公开语料的空白,为世界英语的NLP研究提供支撑资源。

摘要:

语言模型是自然语言处理(NLP)各类应用场景的基础,但在语言变体研究中尚未得到充分应用,即便针对英语这类最普及的语言也是如此。本文是将NLP技术应用于世界英语研究范式的少数初步成果之一,具体工作为构建用于亚洲英语研究的多变体语料库。我们介绍了CCAE(基于中文的亚洲英语语料库)的整体概况,这是一套包含6种基于中文的亚洲英语变体的语料集合。该语料库基于来自6个地区的44.8万份网络文档,共计3.4亿个token。该数据体系将使该语料库成为极具研究潜力的实用资源,可用于亚洲英语研究(尤其是针对目前尚无公开可用语料库的中式英语研究),也是构建变体专属语言模型和开展下游任务的理想数据源,从而为基于NLP的世界英语研究奠定基础。基于该语料库的初步实验也验证了CCAE的实用价值。最后,我们通过此https URL公开了CCAE:https://huggingface.co/datasets/CCAE/CCAE-Corpus。

英文摘要:

Language models have been foundations in various scenarios of NLP applications, but it has not been well applied in language variety studies, even for the most popular language like English. This paper represents one of the few initial efforts to utilize the NLP technology in the paradigm of World Englishes, specifically in creating a multi-variety corpus for studying Asian Englishes. We present an overview of the CCAE -- Corpus of Chinese-based Asian English, a suite of corpora comprising six Chinese-based Asian English varieties. It is based on 340 million tokens in 448 thousand web documents from six regions. The ontology of data would make the corpus a helpful resource with enormous research potential for Asian Englishes (especially for Chinese Englishes for which there has not been a publicly accessible corpus yet so far) and an ideal source for variety-specific language modeling and downstream tasks, thus setting the stage for NLP-based World Englishes studies. And preliminary experiments on this corpus reveal the practical value of CCAE. Finally, we make CCAE available at \href{https://huggingface.co/datasets/CCAE/CCAE-Corpus}{this https URL}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑