IMPACT：基于迭代掩码并行解码的扩散建模文本到音频生成
IMPACT: Iterative Mask-based Parallel Decoding for Text-to-Audio Generation with Diffusion Modeling
AI总结：
本文提出IMPACT框架，通过在连续潜在空间中结合扩散建模与迭代掩码并行解码，在保证音频质量的同时显著降低文本到音频生成的推理延迟，并在AudioCaps上取得最先进性能。
摘要：
文本到音频生成旨在根据自然语言提示合成逼真的声音或音乐。基于扩散的框架，包括Tango和AudioLDM系列，代表了文本到音频生成的最先进水平。尽管实现了高音频保真度，但由于扩散采样过程缓慢，它们会产生显著的推理延迟。MAGNET是一种在离散token上运行的基于掩码的模型，通过迭代掩码并行解码来解决推理速度慢的问题。然而，其音频质量仍落后于基于扩散的模型。在这项工作中，我们提出了IMPACT，一个文本到音频生成框架，在确保快速推理的同时实现音频质量和保真度的高性能。IMPACT在由扩散建模驱动的连续潜在空间中利用迭代掩码并行解码。这种方法消除了离散token的保真度限制，同时保持了有竞争力的推理速度。在AudioCaps上的结果表明，IMPACT在包括Fréchet距离（FD）和Fréchet音频距离（FAD）在内的关键指标上达到了最先进的性能，同时与之前的模型相比显著降低了延迟。项目网站可在https://audio-impact.github.io/获取。
英文摘要：
Text-to-audio generation synthesizes realistic sounds or music given a natural language prompt. Diffusion-based frameworks, including the Tango and the AudioLDM series, represent the state-of-the-art in text-to-audio generation. Despite achieving high audio fidelity, they incur significant inference latency due to the slow diffusion sampling process. MAGNET, a mask-based model operating on discrete tokens, addresses slow inference through iterative mask-based parallel decoding. However, its audio quality still lags behind that of diffusion-based models. In this work, we introduce IMPACT, a text-to-audio generation framework that achieves high performance in audio quality and fidelity while ensuring fast inference. IMPACT utilizes iterative mask-based parallel decoding in a continuous latent space powered by diffusion modeling. This approach eliminates the fidelity constraints of discrete tokens while maintaining competitive inference speed. Results on AudioCaps demonstrate that IMPACT achieves state-of-the-art performance on key metrics including Fréchet Distance (FD) and Fréchet Audio Distance (FAD) while significantly reducing latency compared to prior models. The project website is available at https://audio-impact.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
