arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.11233cs.CL
收藏

RUMLEM:一种基于词典的罗曼什语词形还原器

RUMLEM: A Dictionary-Based Lemmatizer for Romansh

Department of Computational Linguistics, University of Zurich(苏黎世大学计算语言学系)

Dominic P. Fischer, Zachary Hopton, Jannis Vamvas

AI总结:

本文提出RUMLEM,一种覆盖罗曼什语五种主要方言及超区域标准方言的词形还原器,利用社区驱动的词形学数据库,实现77-84%的罗曼什文本词语处理,并展示基于词形还原器的方言识别能力。

摘要:

词形还原——将屈折词形映射到词典形式——是许多自然语言处理应用的关键组成部分。本文提出RUMLEM,一种覆盖罗曼什语五种主要方言及超区域标准方言Rumantsch Grischun的词形还原器。它基于全面的、由社区驱动的罗曼什语形态学数据库,使RUMLEM能够处理典型罗曼什文本中77-84%的词汇。由于每个罗曼什方言都有专门的数据库,RUMLEM的另一个应用是方言意识语言分类。对3万份不同长度的罗曼什文本的评估显示,RUMLEM在95%的情况下正确识别了方言。此外,一个概念证明展示了基于词形还原器的罗曼什语与非罗曼什语语言分类的可行性。

英文摘要:

Lemmatization -- the task of mapping an inflected word form to its dictionary form -- is a crucial component of many NLP applications. In this paper, we present RUMLEM, a lemmatizer that covers the five main varieties of Romansh as well as the supra-regional standard variety Rumantsch Grischun. It is based on comprehensive, community-driven morphological databases for Romansh, enabling RUMLEM to cover 77-84% of the words in a typical Romansh text. Since there is a dedicated database for each Romansh variety, an additional application of RUMLEM is variety-aware language classification. Evaluation on 30'000 Romansh texts of varying lengths shows that RUMLEM correctly identifies the variety in 95% of cases. In addition, a proof of concept demonstrates the feasibility of Romansh vs. non-Romansh language classification based on the lemmatizer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑