处理训练与测试分割不匹配问题:FBK@IWSLT2021
Dealing with training and test segmentation mismatch: FBK@IWSLT2021
AI总结:
本文针对IWSLT 2021离线语音翻译任务中训练与测试数据分割不匹配问题,提出基于Transformer的直接模型,采用知识蒸馏与两步微调流程,并结合自定义混合分割方法,有效缩小了自动与人工分割间的性能差距。
摘要:
本文描述了FBK提交至IWSLT 2021离线语音翻译任务的系统。我们以直接模型参赛,这是一种基于Transformer的架构,训练用于将英语语音音频数据翻译为德语文本。训练流程的特点是采用知识蒸馏和两步微调过程。知识蒸馏和第一步微调均在人工分割的真实数据和合成数据上进行,后者由在可用语料库上训练的MT系统生成。不同的是,第二步微调在MuST-C v2 En-De数据集的随机分割上进行。其主要目的是减少在人工分割数据(即理想的类句子分割)上训练的语音翻译模型,在自动分割音频(即实际、更真实的测试条件)上评估时出现的性能下降。出于同样目的,在将测试数据传递给系统之前,对其应用了考虑音频内容(停顿)和生成分段长度的自定义混合分割过程。在推理时,我们将此过程与基于语音活动检测(VAD)的基线分割方法进行了比较。结果表明了所提出的混合方法的有效性,与人工分割的差距从8.3个BLEU点减少至1.4个BLEU点。
英文摘要:
This paper describes FBK's system submission to the IWSLT 2021 Offline Speech Translation task. We participated with a direct model, which is a Transformer-based architecture trained to translate English speech audio data into German texts. The training pipeline is characterized by knowledge distillation and a two-step fine-tuning procedure. Both knowledge distillation and the first fine-tuning step are carried out on manually segmented real and synthetic data, the latter being generated with an MT system trained on the available corpora. Differently, the second fine-tuning step is carried out on a random segmentation of the MuST-C v2 En-De dataset. Its main goal is to reduce the performance drops occurring when a speech translation model trained on manually segmented data (i.e. an ideal, sentence-like segmentation) is evaluated on automatically segmented audio (i.e. actual, more realistic testing conditions). For the same purpose, a custom hybrid segmentation procedure that accounts for both audio content (pauses) and for the length of the produced segments is applied to the test data before passing them to the system. At inference time, we compared this procedure with a baseline segmentation method based on Voice Activity Detection (VAD). Our results indicate the effectiveness of the proposed hybrid approach, shown by a reduction of the gap with manual segmentation from 8.3 to 1.4 BLEU points.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。