arXiv 2406.073302024-06-11cs.CLcs.AIcs.SDeess.AS

基于CTC的非自回归无文本语音到语音翻译

CTC-based Non-autoregressive Textless Speech-to-Speech Translation

University of Chinese Academy of Sciences（中国科学院大学）；School of Future Science and Engineering, Soochow University（苏州大学未来科学与工程学院）；Key Laboratory of AI Safety, Chinese Academy of Sciences（中国科学院人工智能安全重点实验室）

Qingkai Fang, Zhengrui Ma, Yan Zhou, Min Zhang, Yang Feng

备注：: ACL 2024 Findings

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文研究基于CTC的非自回归语音到语音翻译，结合预训练、知识蒸馏及glancing training等技术，在保持与自回归模型相当翻译质量的同时实现最高26.81倍解码加速。

摘要：

直接语音到语音翻译（S2ST）已取得令人印象深刻的翻译质量，但由于语音序列长度可观，其常常面临解码缓慢的挑战。近年来，一些研究转向非自回归（NAR）模型以加速解码，但翻译质量通常显著落后于自回归（AR）模型。本文研究了基于CTC的NAR模型在S2ST中的表现，因为这些模型已在机器翻译中展现出令人印象深刻的结果。实验结果表明，通过结合预训练、知识蒸馏以及先进的NAR训练技术（如glancing training和非单调潜在对齐），基于CTC的NAR模型取得了与AR模型相当的翻译质量，同时保持高达26.81倍的解码加速。

英文摘要：

Direct speech-to-speech translation (S2ST) has achieved impressive translation quality, but it often faces the challenge of slow decoding due to the considerable length of speech sequences. Recently, some research has turned to non-autoregressive (NAR) models to expedite decoding, yet the translation quality typically lags behind autoregressive (AR) models significantly. In this paper, we investigate the performance of CTC-based NAR models in S2ST, as these models have shown impressive results in machine translation. Experimental results demonstrate that by combining pretraining, knowledge distillation, and advanced NAR training techniques such as glancing training and non-monotonic latent alignments, CTC-based NAR models achieve translation quality comparable to the AR model, while preserving up to 26.81$\times$ decoding speedup.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。