arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.08019cs.SDcs.AIcs.LGeess.ASeess.SP
收藏

通过对抗流匹配优化加速高保真波形生成

Accelerating High-Fidelity Waveform Generation via Adversarial Flow Matching Optimization

Ajou University(亚洲大学);KT Corp.(KT公司);Korea University(高丽大学)

Sang-Hoon Lee, Ha-Yeong Choi, Seong-Whan Lee

AI总结:

本研究提出PeriodWave-Turbo模型,通过对抗流匹配优化改进预训练条件流匹配模型,仅需1000步微调即可将推理步数从16步降至2-4步,扩大骨干网络后在LibriTTS数据集上PESQ达4.454,实现高保真高效波形生成。

摘要:

本文提出了PeriodWave-Turbo,这是一种基于对抗流匹配优化的高保真、高效率波形生成模型。近年来,条件流匹配(CFM)生成模型已成功应用于波形生成任务,其训练仅需单个向量场估计目标。尽管这类模型能够生成高保真波形信号,但与仅需单步生成的基于GAN的模型相比,它们需要的常微分方程(ODE)步数要多得多。此外,由于向量场估计存在噪声,生成的样本往往缺少高频信息,无法保证高频信号的复现。为解决这一局限,我们通过引入固定步长生成器改进,对预训练的基于CFM的生成模型进行增强。我们利用重构损失和对抗反馈来加速高保真波形生成。通过对抗流匹配优化,仅需1000步微调即可在多种客观指标上达到当前最优性能。此外,我们将推理速度从16步大幅降低至2步或4步。另外,通过将PeriodWave的骨干网络参数量从29M扩大到70M以提升泛化能力,PeriodWave-Turbo取得了前所未有的性能,在LibriTTS数据集上的语音质量感知评估(PESQ)得分为4.454。音频样本、源代码和模型检查点将发布于https://github.com/sh-lee-prml/PeriodWave。

英文摘要:

This paper introduces PeriodWave-Turbo, a high-fidelity and high-efficient waveform generation model via adversarial flow matching optimization. Recently, conditional flow matching (CFM) generative models have been successfully adopted for waveform generation tasks, leveraging a single vector field estimation objective for training. Although these models can generate high-fidelity waveform signals, they require significantly more ODE steps compared to GAN-based models, which only need a single generation step. Additionally, the generated samples often lack high-frequency information due to noisy vector field estimation, which fails to ensure high-frequency reproduction. To address this limitation, we enhance pre-trained CFM-based generative models by incorporating a fixed-step generator modification. We utilized reconstruction losses and adversarial feedback to accelerate high-fidelity waveform generation. Through adversarial flow matching optimization, it only requires 1,000 steps of fine-tuning to achieve state-of-the-art performance across various objective metrics. Moreover, we significantly reduce inference speed from 16 steps to 2 or 4 steps. Additionally, by scaling up the backbone of PeriodWave from 29M to 70M parameters for improved generalization, PeriodWave-Turbo achieves unprecedented performance, with a perceptual evaluation of speech quality (PESQ) score of 4.454 on the LibriTTS dataset. Audio samples, source code and checkpoints will be available at https://github.com/sh-lee-prml/PeriodWave.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑