One crucial challenge of real-world multilingual speech recognition is the long-tailed distribution problem, where some resource-rich languages like English have abundant training data, but a long tail of low-resource languages have varying amounts of limited training data. To overcome the long-tail problem, in this paper, we propose Adapt-and-Adjust (A2), a transformer-based multi-task learning framework for end-to-end multilingual speech recognition. The A2 framework overcomes the long-tail problem via three techniques: (1) exploiting a pretrained multilingual language model (mBERT) to improve the performance of low-resource languages; (2) proposing dual adapters consisting of both language-specific and language-agnostic adaptation with minimal additional parameters; and (3) overcoming the class imbalance, either by imposing class priors in the loss during training or adjusting the logits of the softmax output during inference. Extensive experiments on the CommonVoice corpus show that A2 significantly outperforms conventional approaches.
Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervision
School of Computer Science and Technology, Xinjiang University（新疆大学计算机科学与技术学院）；China Unicom (Guangdong) Industrial Internet Co., Ltd（中国联通（广东）工业互联网有限公司）；Tsinghua University（清华大学）
Google Research（谷歌研究院）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
In this paper, we explore the challenging problem of performing a generative task in a target language when labeled data is only available in English, using summarization as a case study. We assume a strict setting with no access to parallel data or machine translation and find that common transfer learning approaches struggle in this setting, as a generative multilingual model fine-tuned purely on English catastrophically forgets how to generate non-English. Given the recent rise of parameter-efficient adaptation techniques, we conduct the first investigation into how one such method, prompt …
Distributed representations of meaning are a natural way to encode covariance relationships between words and phrases in NLP. By overcoming data sparsity problems, as well as providing information about semantic relatedness which is not available in discrete representations, distributed representations have proven useful in many NLP tasks. Recent work has shown how compositional semantic representations can successfully be applied to a number of monolingual applications such as sentiment analysis. At the same time, there has been some initial success in work on learning shared word-level repre…
Monash University（莫纳什大学）；Huawei Technologies Co Ltd（华为技术有限公司）
Learning multilingual and multi-domain translation model is challenging as the heterogeneous and imbalanced data make the model converge inconsistently over different corpora in real world. One common practice is to adjust the share of each corpus in the training, so that the learning process is balanced and low-resource cases can benefit from the high resource ones. However, automatic balancing methods usually depend on the intra- and inter-dataset characteristics, which is usually agnostic or requires human priors. In this work, we propose an approach, MultiUAT, that dynamically adjusts the …
Facebook AI Research（Facebook AI研究院）；Inria（法国国家信息与自动化研究所）；LORIA（洛林计算机科学实验室）
Progress in sentence simplification has been hindered by a lack of labeled parallel simplification data, particularly in languages other than English. We introduce MUSS, a Multilingual Unsupervised Sentence Simplification system that does not require labeled simplification data. MUSS uses a novel approach to sentence simplification that trains strong models using sentence-level paraphrase data instead of proper simplification data. These models leverage unsupervised pretraining and controllable generation mechanisms to flexibly adjust attributes such as length and lexical complexity at inferen…
Beihang University（北京航空航天大学）；University of Chinese Academy of Sciences（中国科学院大学）
Although the multilingual Neural Machine Translation(NMT), which extends Google's multilingual NMT, has ability to perform zero-shot translation and the iterative self-learning algorithm can improve the quality of zero-shot translation, it confronts with two problems: the multilingual NMT model is prone to generate wrong target language when implementing zero-shot translation; the self-learning algorithm, which uses beam search to generate synthetic parallel data, demolishes the diversity of the generated source language and amplifies the impact of the same noise during the iterative learning …
In this work, we take the first steps towards building a universal rewriter: a model capable of rewriting text in any language to exhibit a wide variety of attributes, including styles and languages, while preserving as much of the original semantics as possible. In addition to obtaining state-of-the-art results on unsupervised translation, we also demonstrate the ability to do zero-shot sentiment transfer in non-English languages using only English exemplars for sentiment. We then show that our model is able to modify multiple attributes at once, for example adjusting both language and sentim…
Indic-TunedLens: Interpreting Multilingual Models in Indian Languages
Dwarkadas Jivanlal Sanghvi College of Engineering（达沃拉斯·吉文拉尔工程学院）；Indian Institute of Technology Jodhpur（印度理工学院乔浦尔分校）；King’s College London（伦敦国王学院）；Indian Institute of Technology Patna（印度理工学院帕特纳分校）
The great majority of languages in the world are considered under-resourced for the successful application of deep learning methods. In this work, we propose a meta-learning approach to document classification in limited-resource setting and demonstrate its effectiveness in two different settings: few-shot, cross-lingual adaptation to previously unseen languages; and multilingual joint training when limited target-language data is available during training. We conduct a systematic comparison of several meta-learning methods, investigate multiple settings in terms of data availability and show …
MEMLA: Enhancing Multilingual Knowledge Editing with Neuron-Masked Low-Rank Adaptation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Output Language Confusion under Multilingual Prompt Contamination
University of Tübingen（蒂宾根大学）；Artificial Intelligence Institute, University of South Carolina（南卡罗来纳大学人工智能研究所）；Thapar Institute of Engineering & Technology（塔帕尔工程技术学院）；Indian Institute of Technology Kanpur（印度理工学院坎普尔分校）；Indian AI Research Organization（印度人工智能研究组织）
Overcoming Language Priors for Visual Question Answering Based on Knowledge Distillation
School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；Joint Laboratory of HUST and Pingan Property & Casualty Research (HPL)（华中科技大学与平安财产保险研究联合实验室）
MoCE: Adaptive Mixture of Contextualization Experts for Byte-based Neural Machine Translation
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Key Laboratory of AI Safety, Chinese Academy of Sciences（中国科学院人工智能安全重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Washington University in St. Louis（圣路易斯华盛顿大学）
