构建历史土耳其语自然语言处理的基础:资源与模型
Building Foundations for Natural Language Processing of Historical Turkish: Resources and Models
AI总结:
本文提出历史土耳其语NLP的基础资源和模型,包括首个命名实体识别数据集HisTR和首个通用依赖树库OTA-BOUN,以及基于Transformer的模型,提升了历史土耳其语的计算分析能力。
摘要:
本文介绍了用于历史土耳其语自然语言处理(NLP)的基础资源和模型,这是一个在计算语言学中长期被忽视的领域。我们提出了首个命名实体识别(NER)数据集HisTR和首个通用依赖树库OTA-BOUN,以及基于Transformer的模型,这些模型利用这些数据集进行命名实体识别、依赖解析和词性标注任务。此外,我们引入了奥斯曼文本语料库(OTC),这是一个干净的转写历史土耳其语文本语料库,涵盖了广泛的历史时期。我们的实验结果展示了在历史土耳其语的计算分析中显著的改进,实现了需要理解历史语言结构的任务的强性能——具体而言,命名实体识别达到90.29%的F1值,依赖解析达到73.79%的LAS,词性标注达到94.98%的F1值。它们还突显了现有挑战,如领域适应和不同时期的语言变体。所有提出的资源和模型均可在https://hf.co/bucolin获取,以作为未来历史土耳其语NLP进步的基准。
英文摘要:
This paper introduces foundational resources and models for natural language processing (NLP) of historical Turkish, a domain that has remained underexplored in computational linguistics. We present the first named entity recognition (NER) dataset, HisTR, and the first Universal Dependencies treebank, OTA-BOUN, for a historical form of the Turkish language along with transformer-based models trained using these datasets for named entity recognition, dependency parsing, and part-of-speech tagging tasks. Furthermore, we introduce the Ottoman Text Corpus (OTC), a clean corpus of transliterated historical Turkish texts that spans a wide range of historical periods. Our experimental results demonstrate prominent improvements in the computational analysis of historical Turkish, achieving strong performance on tasks that require understanding of historical linguistic structures -- specifically, 90.29% F1 in named entity recognition, 73.79% LAS for dependency parsing, and 94.98% F1 for part-of-speech tagging. They also highlight existing challenges, such as domain adaptation and language variations between time periods. All the resources and models presented are available at https://hf.co/bucolin to serve as a benchmark for future progress in historical Turkish NLP.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。