arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.22110cs.CL
收藏

拆分纳瓦特尔语神经机器翻译:针对马塞瓦科帕变体的单变体专业化

Unfragmenting Nawatl NMT: Single-Variant Specialization for Masewalkopa

University of Colorado Boulder(科罗拉多大学博尔德分校)

Luis A. Mercado-Campos, Alexis Palmer

AI总结:

针对纳瓦特尔语的变体碎片化问题,研究将mT5-large专门化到马塞瓦科帕变体,结合定制分析器与LoRA、CTranslate2量化技术,提升翻译性能并降低部署延迟。

摘要:

在少数群体(低资源)语言技术中,应对数据稀疏性的常见做法是聚合多种变体的文本,并将合并后的数据视为单一语料库。然而,密切相关的本土变体混合语料库会产生一种人工语域,它不对应任何单一说话者群体,我们将这种结构缺陷称为“变体碎片化”。这一问题在墨西哥本土的少数群体语言(如纳瓦特尔语)中尤为突出。基于包含70000对的多变体基础数据,我们将mT5-large检查点专门应用于圣安娜特拉科滕科使用的纳瓦特尔语变体马塞瓦科帕(nhn)。我们整理了2665个平行对,保留所有句子级数据用于评估,并通过非对称回译纳入192条由长者转录的目标句子。定制的有限状态分析器将未分析的表层形式占比从82.0%降至19.9%,并提供功能句法标签作为纯文本前缀,使模型能够复用现有的子词嵌入,而非依赖冻结的特殊标记。最终得到的LoRA-v9c模型在开发集上取得58.46的chrF++分数,在定向测试集上取得58.20/57.90的chrF++分数。最后,CTranslate2的int8量化将CPU延迟从每请求137.5秒降至1.8-2.6秒,支持成本可控的Cloud Run部署。

英文摘要:

In minoritized (low-resource) language technologies, a common response to data sparsity is to aggregate text across multiple varieties and treat the combined data as a single corpus. However, mixed corpora of closely related indigenous variants can produce an artificial register that is accountable to no single speaker community, a structural failure we call variant fragmentation. This issue is acute for minoritized languages native to Mexico, such as Nawatl. Building on a 70,000-pair multi-variant foundation, we specialize an mT5-large checkpoint to Masewalkopa (nhn), the Nawatl variety spoken in Santa Ana Tlacotenco. We curate 2,665 parallel pairs, reserve all sentence-level data for evaluation, and incorporate 192 elder-transcribed target sentences via asymmetric backtranslation. A custom finite-state analyzer reduces unanalyzed surface forms from 82.0% to 19.9% and supplies functional syntax tags as plain-text prefixes, allowing the model to reuse existing subword embeddings rather than relying on frozen special tokens. The resulting LoRA-v9c-b model achieves 58.46 dev chrF++ and 58.20/57.90 directional test chrF++. Finally, CTranslate2 int8 quantization reduces CPU latency from 137.5 to 1.8-2.6 seconds per request, enabling cost-bounded Cloud Run deployment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑