arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.13016cs.CLcs.LG
收藏

Darija 开放数据集的演进:推出第二版

The Evolution of Darija Open Dataset: Introducing Version 2

Aissam Outchakoucht, Hamza Es-Samaali

AI总结:

本文介绍 Darija 开放数据集(DODa)第二版,该大规模开源数据集旨在提升摩洛哥方言的自然语言处理能力,通过提供约十万条带语义句法标注及多语种书写的翻译条目,支持社区语言应用并展望未来扩展。

摘要:

Darija 开放数据集(DODa)是一个开源项目,旨在提升摩洛哥方言 Darija 的自然语言处理能力。DODa 拥有约 100,000 个条目,是同类项目中规模最大的 Darija-英语翻译协作项目。该数据集包含语义和句法分类、拼写变体、多种时态的动词变位,以及数万条翻译句子。数据集中的条目同时使用拉丁字母和阿拉伯字母书写,反映了不同来源和应用中的语言变体和偏好。此类数据集的可用性对于开发能够准确理解和生成 Darija 的应用程序至关重要,从而支持摩洛哥社区的语言需求,并可能扩展到邻近地区的类似方言。本文探讨了 DODa 的战略重要性、其当前成就,以及未来预期的增强功能,这些功能将继续推动其在全球自然语言处理领域的应用和扩展。

英文摘要:

Darija Open Dataset (DODa) represents an open-source project aimed at enhancing Natural Language Processing capabilities for the Moroccan dialect, Darija. With approximately 100,000 entries, DODa stands as the largest collaborative project of its kind for Darija-English translation. The dataset features semantic and syntactic categorizations, variations in spelling, verb conjugations across multiple tenses, as well as tens of thousands of translated sentences. The dataset includes entries written in both Latin and Arabic alphabets, reflecting the linguistic variations and preferences found in different sources and applications. The availability of such dataset is critical for developing applications that can accurately understand and generate Darija, thus supporting the linguistic needs of the Moroccan community and potentially extending to similar dialects in neighboring regions. This paper explores the strategic importance of DODa, its current achievements, and the envisioned future enhancements that will continue to promote its use and expansion in the global NLP landscape.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑