Central Yup'ik and Machine Translation of Low-Resource Polysynthetic Languages

Stanford University（斯坦福大学）；Salesforce Research（Salesforce 研究院）

Christopher Liu, Laura Dominé, Kevin Chavez, Richard Socher

英文摘要：

Machine translation tools do not yet exist for the Yup'ik language, a polysynthetic language spoken by around 8,000 people who live primarily in Southwest Alaska. We compiled a parallel text corpus for Yup'ik and English and developed a morphological parser for Yup'ik based on grammar rules. We trained a seq2seq neural machine translation model with attention to translate Yup'ik input into English. We then compared the influence of different tokenization methods, namely rule-based, unsupervised (byte pair encoding), and unsupervised morphological (Morfessor) parsing, on BLEU score accuracy for Yup'ik to English translation. We find that using tokenized input increases the translation accuracy compared to that of unparsed input. Although overall Morfessor did best with a vocabulary size of 30k, our first experiments show that BPE performed best with a reduced vocabulary size.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Neural Polysynthetic Language Modelling

École de Technologie Supérieure（高等技术学院）；Johns Hopkins University（约翰斯·霍普金斯大学）；Amazon（亚马逊）；Facebook（脸书）；Google（谷歌）；Microsoft（微软）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；National Research University Higher School of Economics (HSE University)（国立研究型大学高等经济学院（HSE大学））；Indiana University Bloomington（印第安纳大学伯明顿分校）；Carnegie Mellon University（卡内基梅隆大学）

Research in natural language processing commonly assumes that approaches that work well for English and and other widely-used languages are "language agnostic". In high-resource languages, especially those that are analytic, a common approach is to treat morphologically-distinct variants of a common root as completely independent word types. This assumes, that there are limited morphological inflections per root, and that the majority will appear in a large enough corpus, so that the model can adequately learn statistics about each form. Approaches like stemming, lemmatization, or subword segm…

2020-05-13cs.CL

Lost in Translation: Analysis of Information Loss During Machine Translation Between Polysynthetic and Fusional Languages

Instituto de Investigaciones en Matemáticas Aplicadas y en Sistemas, Universidad Nacional Autónoma de México（墨西哥国立自治大学应用数学与系统科学研究所）；Facultad de Estudios Superiores Acatlán, Universidad Nacional Autónoma de México（墨西哥国立自治大学阿卡特朗高等研究学院）；Centro de Estudios Lingüísticos y Literarios, El Colegio de México（墨西哥学院语言与文学研究中心）；Center for Data Science, New York University（纽约大学数据科学中心）

Machine translation from polysynthetic to fusional languages is a challenging task, which gets further complicated by the limited amount of parallel text available. Thus, translation performance is far from the state of the art for high-resource and more intensively studied language pairs. To shed light on the phenomena which hamper automatic translation to and from polysynthetic languages, we study translations from three low-resource, polysynthetic languages (Nahuatl, Wixarika and Yorem Nokki) into Spanish and vice versa. Doing so, we find that in a morpheme-to-morpheme alignment an importan…

2018-07-01cs.CL

BPE vs. Morphological Segmentation: A Case Study on Machine Translation of Four Polysynthetic Languages

University of Stuttgart（斯图加特大学）；Universidad Nacional Autónoma de México（墨西哥国立自治大学）；University of Edinburgh（爱丁堡大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）

Morphologically-rich polysynthetic languages present a challenge for NLP systems due to data sparsity, and a common strategy to handle this issue is to apply subword segmentation. We investigate a wide variety of supervised and unsupervised morphological segmentation methods for four polysynthetic languages: Nahuatl, Raramuri, Shipibo-Konibo, and Wixarika. Then, we compare the morphologically inspired segmentation methods against Byte-Pair Encodings (BPEs) as inputs for machine translation (MT) when translating to and from Spanish. We show that for all language pairs except for Nahuatl, an uns…

2022-03-16cs.AIcs.CL

A Resource for Computational Experiments on Mapudungun

Carnegie Mellon University（卡内基梅隆大学）；Wake Forest University（维克森林大学）

We present a resource for computational experiments on Mapudungun, a polysynthetic indigenous language spoken in Chile with upwards of 200 thousand speakers. We provide 142 hours of culturally significant conversations in the domain of medical treatment. The conversations are fully transcribed and translated into Spanish. The transcriptions also include annotations for code-switching and non-standard pronunciations. We also provide baseline results on three core NLP tasks: speech recognition, speech synthesis, and machine translation between Spanish and Mapudungun. We further explore other app…

2020-04-05cs.CL

Finetuning a Kalaallisut-English machine translation system using web-crawled data

Dartmouth College（达特茅斯学院）

West Greenlandic, known by native speakers as Kalaallisut, is an extremely low-resource polysynthetic language spoken by around 56,000 people in Greenland. Here, we attempt to finetune a pretrained Kalaallisut-to-English neural machine translation (NMT) system using web-crawled pseudoparallel sentences from around 30 multilingual websites. We compile a corpus of over 93,000 Kalaallisut sentences and over 140,000 Danish sentences, then use cross-lingual sentence embeddings and approximate nearest-neighbors search in an attempt to mine near-translations from these corpora. Finally, we translate …

2022-06-05cs.AIcs.CL

Quantifying Synthesis and Fusion and their Impact on Machine Translation

University of Edinburgh（爱丁堡大学）；New York University（纽约大学）；Aalborg University（奥尔堡大学）

Theoretical work in morphological typology offers the possibility of measuring morphological diversity on a continuous scale. However, literature in Natural Language Processing (NLP) typically labels a whole language with a strict type of morphology, e.g. fusional or agglutinative. In this work, we propose to reduce the rigidity of such claims, by quantifying morphological typology at the word and segment level. We consider Payne (2017)'s approach to classify morphology using two indices: synthesis (e.g. analytic to polysynthetic) and fusion (agglutinative to fusional). For computing synthesis…

2022-05-06cs.AIcs.CL