arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.07015cs.CL
收藏

语料去重或重复在自然语言处理中少数资源语言的应用?一个案例研究:墨西哥的纳瓦特尔语

Corpora deduplication or duplication in Natural Language Processing of few resourced languages ? A case of study: The Mexico's Nahuatl

LIFAT, Université de Tours(图尔大学计算机科学、人工智能与图像处理实验室);Instituto de Investigaciones en Educación, Universidad Veracruzana(韦拉克鲁斯大学教育研究所);Trading Central(Trading Central公司);Independent Researcher(独立研究员)

Juan-José Guzman-Landa, Juan-Manuel Torres-Moreno, Graham Ranger, Miguel Figueroa-Saavedra, Martha-Lorena Avendaño-Garrido, Elvys Linhares-Pontes, Luis-Gil Moreno-Jiménez

AI总结:

本文探讨在计算资源有限的语言中,语料重复是否有助于自然语言处理。通过纳瓦特尔语语料扩展实验,发现可控重复可提升嵌入学习效果。

摘要:

本文旨在回答在计算资源有限的语言中,数据重复是否对自然语言处理有益。针对纳瓦特尔语,一种聚合性和多合成性的π语言,通过可控重复扩展语料,提升嵌入学习效果。实验使用增量重复技术,训练静态嵌入并在句子级语义相似性任务中评估。结果表明,与未扩展语料相比,增量重复在性能上有所提升,且该技术尚未在文献中被使用。

英文摘要:

In this article, we seek to answer the following question: could data duplication be useful in Natural Language Processing (NLP) for languages with limited computational resources? In this type of languages (or $π$-languages), corpora available for training Large Language Models are virtually non-existent. In particular, we will study the impact of corpora expansion in Nawatl, an agglutinative and polysynthetic $π$-language spoken by over 2 million people, with a large number of dialectal varieties. The aim is to expand the new $π$-yalli corpus, which contains a limited number of Nawatl texts, by duplicating it in a controlled way. In our experiments, we will use the incremental duplication technique. The aim is to learn embeddings that are well-suited to NLP tasks. Thus, static embeddings were trained and evaluated in a sentence-level semantic similarity task. Our results show a moderate improvement in performance when using incremental duplication compared to the results obtained using only the corpus without expansion. Furthermore, to our knowledge, this technique has not yet been used in the literature.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑