In this paper, we address the task of improving pair-wise machine translation for specific low resource Indian languages. Multilingual NMT models have demonstrated a reasonable amount of effectiveness on resource-poor languages. In this work, we show that the performance of these models can be significantly improved upon by using back-translation through a filtered back-translation process and subsequent fine-tuning on the limited pair-wise language corpora. The analysis in this paper suggests that this method can significantly improve a multilingual model's performance over its baseline, yielding state-of-the-art results for various Indian languages.
Delhi Technological University (Formerly Delhi college of Engineering)（德里理工大学（原德里工程学院））
A large number of significant assets are available online in English, which is frequently translated into native languages to ease the information sharing among local people who are not much familiar with English. However, manual translation is a very tedious, costly, and time-taking process. To this end, machine translation is an effective approach to convert text to a different language without any human involvement. Neural machine translation (NMT) is one of the most proficient translation techniques amongst all existing machine translation systems. In this paper, we have applied NMT on two…
IIIT Hyderabad（国际信息技术研究所海得拉巴分校）；University of Bath（巴斯大学）
Indian language machine translation performance is hampered due to the lack of large scale multi-lingual sentence aligned corpora and robust benchmarks. Through this paper, we provide and analyse an automated framework to obtain such a corpus for Indian language neural machine translation (NMT) systems. Our pipeline consists of a baseline NMT system, a retrieval module, and an alignment module that is used to work with publicly available websites such as press releases by the government. The main contribution towards this effort is to obtain an incremental method that uses the above pipeline t…
Pune Institute of Computer Technology（浦那计算机技术学院）；Indian Institute of Technology Madras（印度理工学院马德拉斯分校）
Recent advancements in Neural Machine Translation (NMT) models have proved to produce a state of the art results on machine translation for low resource Indian languages. This paper describes the neural machine translation systems for the English-Hindi language presented in AdapMT Shared Task ICON 2020. The shared task aims to build a translation system for Indian languages in specific domains like Artificial Intelligence (AI) and Chemistry using a small in-domain parallel corpus. We evaluated the effectiveness of two popular NMT models i.e, LSTM, and Transformer architectures for the English-…
Impact of Visual Context on Noisy Multimodal NMT: An Empirical Study for English to Indian Languages
Indian Institute of Technology Patna India；Indian Institute of Technology Jodhpur \& Indian Institute of Technology Patna India；Indian Institute of Technology Patna；Indian Institute of Technology Jodhpur \& Indian Institute of Technology Patna
IIT Bombay（印度理工学院孟买分校）；IITB-Monash Research Academy（印度理工学院孟买分校-莫纳什大学联合研究学院）；Monash University（莫纳什大学）；NICT（日本情报通信研究机构）
Cognates are variants of the same lexical form across different languages; for example 'fonema' in Spanish and 'phoneme' in English are cognates, both of which mean 'a unit of sound'. The task of automatic detection of cognates among any two languages can help downstream NLP tasks such as Cross-lingual Information Retrieval, Computational Phylogenetics, and Machine Translation. In this paper, we demonstrate the use of cross-lingual word embeddings for detecting cognates among fourteen Indian Languages. Our approach introduces the use of context from a knowledge graph to generate improved featu…
Georgia Institute of Technology（佐治亚理工学院）；ByteDance（字节跳动）
Many multi-domain neural machine translation (NMT) models achieve knowledge transfer by enforcing one encoder to learn shared embedding across domains. However, this design lacks adaptation to individual domains. To overcome this limitation, we propose a novel multi-domain NMT model using individual modules for each domain, on which we apply word-level, adaptive and layer-wise domain mixing. We first observe that words in a sentence are often related to multiple domains. Hence, we assume each word has a domain proportion, which indicates its domain preference. Then word representations are obt…
CorIL: Towards Enriching Indian Language to Indian Language Parallel Corpora and Machine Translation Systems
Department of Computer Science and Engineering, Indian Institute of Technology Patna（计算机科学与工程系，印度理工学院帕纳布）；SNLP Lab, CDAC Noida（SNLP实验室，CDAC诺伊达）；Department of Computer Science and Engineering, Manipal Institute of Technology（计算机科学与工程系，曼尼普尔理工学院）；Department of Computer Science and Engineering, Dhirubhai Ambani University, Gandhinagar（计算机科学与工程系，迪鲁布希阿米大学，甘地 Nagar）；LTRC, IIIT Hyderabad（LTRC，IIIT海得拉尔）；Department of CSE, IIIT Bhubaneshwar（CSE系，IIIT布巴内什瓦尔）；Department of AI, SVNIT, Surat（人工智能系，SVNIT，萨拉特）；Department of Linguistics, University of Kashmir（语言学系，克什米尔大学）；VLSI Design Group, CDAC Bangalore（VLSI设计组，CDAC班加罗尔）；Department of CSE, Punjabi University（CSE系，旁遮普大学）；Department of CSE, Govt. College for Women Jammu（CSE系，贾姆穆政府女子学院）
With the rapid development of artificial intelligence (AI), there is a trend in moving AI applications, such as neural machine translation (NMT), from cloud to mobile devices. Constrained by limited hardware resources and battery, the performance of on-device NMT systems is far from satisfactory. Inspired by conditional computation, we propose to improve the performance of on-device NMT systems with dynamic multi-branch layers. Specifically, we design a layer-wise dynamic multi-branch network with only one branch activated during training and inference. As not all branches are activated during…
Zhengzhou University（郑州大学）；DFKI（德国人工智能研究中心）；Saarland University（萨尔大学）；China Mobile Online Services（中国移动在线服务公司）；College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）
Existing curriculum learning approaches to Neural Machine Translation (NMT) require sampling sufficient amounts of "easy" samples from training data at the early training stage. This is not always achievable for low-resource languages where the amount of training data is limited. To address such limitation, we propose a novel token-wise curriculum learning approach that creates sufficient amounts of easy samples. Specifically, the model learns to predict a short sub-sequence from the beginning part of each target sentence at the early stage of training, and then the sub-sequence is gradually e…
Training neural machine translation (NMT) models in federated learning (FL) settings could be inefficient both computationally and communication-wise, due to the large size of translation engines as well as the multiple rounds of updates required to train clients and a central server. In this paper, we explore how to efficiently build NMT models in an FL setup by proposing a novel solution. In order to reduce the communication overhead, out of all neural layers we only exchange what we term "Controller" layers. Controllers are a small number of additional neural components connected to our pre…
We present effective pre-training strategies for neural machine translation (NMT) using parallel corpora involving a pivot language, i.e., source-pivot and pivot-target, leading to a significant improvement in source-target translation. We propose three methods to increase the relation among source, pivot, and target languages in the pre-training: 1) step-wise training of a single model for different language pairs, 2) additional adapter component to smoothly connect pre-trained encoder and decoder, and 3) cross-lingual encoder training via autoencoding of the pivot language. Our methods great…
University of Chinese Academy of Sciences（中国科学院大学）
Cross-attention is an important component of neural machine translation (NMT), which is always realized by dot-product attention in previous methods. However, dot-product attention only considers the pair-wise correlation between words, resulting in dispersion when dealing with long sentences and neglect of source neighboring relationships. Inspired by linguistics, the above issues are caused by ignoring a type of cross-attention, called concentrated attention, which focuses on several central words and then spreads around them. In this work, we apply Gaussian Mixture Model (GMM) to model the …
IRIS: Interleaved Reinforcement with Incremental Staged Curriculum for Cross-Lingual Mathematical Reasoning
Singapore Institute of Technology（新加坡理工学院）；IIIT Delhi（德里印度理工学院）；University of California, San Diego（加州大学圣地亚哥分校）；Nanyang Technological University（南洋理工大学）；NVIDIA
Indic-TunedLens: Interpreting Multilingual Models in Indian Languages
Dwarkadas Jivanlal Sanghvi College of Engineering（达沃拉斯·吉文拉尔工程学院）；Indian Institute of Technology Jodhpur（印度理工学院乔浦尔分校）；King’s College London（伦敦国王学院）；Indian Institute of Technology Patna（印度理工学院帕特纳分校）
