TransFusion:用多项式扩散进行语音转录
TransFusion: Transcribing Speech with Multinomial Diffusion
摘要:
扩散模型在图像合成域中显示出特殊的缩放特性,并且最初的尝试显示出将扩散应用于无条件文本合成的类似益处。去噪扩散模型试图迭代地细化采样的噪声信号,直到它类似于相干信号(例如图像或书面句子)。在这项工作中,我们的目标是看看扩散模型的好处是否也可以实现语音识别。为此,我们提出了一种新的方式来执行语音识别使用扩散模型的条件下,预先训练的语音功能。具体来说,我们提出了TransFusion:一种转录扩散模型,它将随机字符序列迭代去噪为与条件话语的转录相对应的连贯文本。我们展示了与现有的高性能对比模型LibriSpeech语音识别基准相当的性能。据我们所知,我们是第一个将去噪扩散应用于语音识别的。我们还提出了新的技术,有效地采样和解码多项式扩散模型。这些是必需的,因为传统的方法从声学模型采样是不可能的,我们的新的离散扩散方法。代码和训练模型可在以下网址获得:https://github.com/RF5/transfusion-asr
英文摘要:
Diffusion models have shown exceptional scaling properties in the image synthesis domain, and initial attempts have shown similar benefits for applying diffusion to unconditional text synthesis. Denoising diffusion models attempt to iteratively refine a sampled noise signal until it resembles a coherent signal (such as an image or written sentence). In this work we aim to see whether the benefits of diffusion models can also be realized for speech recognition. To this end, we propose a new way to perform speech recognition using a diffusion model conditioned on pretrained speech features. Specifically, we propose TransFusion: a transcribing diffusion model which iteratively denoises a random character sequence into coherent text corresponding to the transcript of a conditioning utterance. We demonstrate comparable performance to existing high-performing contrastive models on the LibriSpeech speech recognition benchmark. To the best of our knowledge, we are the first to apply denoising diffusion to speech recognition. We also propose new techniques for effectively sampling and decoding multinomial diffusion models. These are required because traditional methods of sampling from acoustic models are not possible with our new discrete diffusion approach. Code and trained models are available: https://github.com/RF5/transfusion-asr
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。