“听、懂、译”：三重监管让端到端语篇翻译脱钩
"Listen, Understand and Translate": Triple Supervision Decouples End-to-end Speech-to-text Translation
摘要：
端到端语音到文本翻译（ST）获取源语言的音频并以目标语言输出文本。现有的方法受到并行语料库数量的限制。我们能否建立一个系统来充分利用平行ST语料库中的信号？人类的理解系统由听觉感知和认知加工两部分组成，这给我们的启示是：在本文中，我们提出了Listen-Understand-Translate（LUT），这是一个具有三重监督信号的统一框架，可以解耦端到端的语音到文本翻译任务。LUT能够引导声学编码器从听觉输入中提取尽可能多的信息。此外，LUT利用预训练的BERT模型来强制上层编码器产生尽可能多的语义信息，而不需要额外的数据。我们在一组不同的语音翻译基准上进行实验，包括Librispeech英语-法语，IWITOS英语-德语和TED英语-汉语。我们的结果表明LUT实现了最先进的性能，优于以前的方法。该代码可在https://github.com/dqqcasia/st上获得。
英文摘要：
An end-to-end speech-to-text translation (ST) takes audio in a source language and outputs the text in a target language. Existing methods are limited by the amount of parallel corpus. Can we build a system to fully utilize signals in a parallel ST corpus? We are inspired by human understanding system which is composed of auditory perception and cognitive processing. In this paper, we propose Listen-Understand-Translate, (LUT), a unified framework with triple supervision signals to decouple the end-to-end speech-to-text translation task. LUT is able to guide the acoustic encoder to extract as much information from the auditory input. In addition, LUT utilizes a pre-trained BERT model to enforce the upper encoder to produce as much semantic information as possible, without extra data. We perform experiments on a diverse set of speech translation benchmarks, including Librispeech English-French, IWSLT English-German and TED English-Chinese. Our results demonstrate LUT achieves the state-of-the-art performance, outperforming previous methods. The code is available at https://github.com/dqqcasia/st.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
