低资源语言文本规范化：利古里亚语案例
Text normalization for low-resource languages: the case of Ligurian
AI总结：
针对低资源语言利古里亚语，构建首个开源语料库，用紧凑Transformer模型结合反向翻译与适当分词，实现低错误率文本规范化。
摘要：
文本规范化对于缺乏严格拼写规范或经历多次拼写改革的低资源语言是一项关键技术。迄今为止，低资源文本规范化依赖于手工规则，这些规则被认为比神经方法更具数据效率。本文研究了利古里亚语（一种濒危罗曼语）的文本规范化案例。我们收集了4,394个利古里亚语句子及其规范化版本，以及首个开源利古里亚语单语语料库。我们证明，尽管可用数据量很小，通过使用反向翻译和适当的tokenization，可以训练一个紧凑的基于Transformer的模型，实现极低的错误率。
英文摘要：
Text normalization is a crucial technology for low-resource languages which lack rigid spelling conventions or that have undergone multiple spelling reforms. Low-resource text normalization has so far relied upon hand-crafted rules, which are perceived to be more data efficient than neural methods. In this paper we examine the case of text normalization for Ligurian, an endangered Romance language. We collect 4,394 Ligurian sentences paired with their normalized versions, as well as the first open source monolingual corpus for Ligurian. We show that, in spite of the small amounts of data available, a compact transformer-based model can be trained to achieve very low error rates by the use of backtranslation and appropriate tokenization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
