双工扩散模型改进语音到语音翻译
Duplex Diffusion Models Improve Speech-to-Speech Translation
AI总结：
本文提出双工扩散模型，通过将扩散概率模型应用于可逆双工Conformer两侧，首次实现可逆语音翻译，显著提升ASR-BLEU分数。
摘要：
语音到语音翻译是一个典型的序列到序列学习任务，自然具有两个方向。如何有效利用双向监督信号为两个方向生成高保真音频？现有方法要么训练两个独立模型，要么采用多任务学习模型，效率低且性能较差。本文提出双工扩散模型，将扩散概率模型应用于可逆双工Conformer的两侧，使任一端能同时输入和输出不同语言的语音。该模型通过翻转输入输出端实现可逆语音翻译。实验表明，该模型首次成功实现可逆语音翻译，与一系列最先进基线相比，ASR-BLEU分数显著提升。
英文摘要：
Speech-to-speech translation is a typical sequence-to-sequence learning task that naturally has two directions. How to effectively leverage bidirectional supervision signals to produce high-fidelity audio for both directions? Existing approaches either train two separate models or a multitask-learned model with low efficiency and inferior performance. In this paper, we propose a duplex diffusion model that applies diffusion probabilistic models to both sides of a reversible duplex Conformer, so that either end can simultaneously input and output a distinct language's speech. Our model enables reversible speech translation by simply flipping the input and output ends. Experiments show that our model achieves the first success of reversible speech translation with significant improvements of ASR-BLEU scores compared with a list of state-of-the-art baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
