无教师自蒸馏一致性轨迹学习用于快速语音增强

中文摘要 AI 辅助

一致性轨迹模型为快速、高质量的语音增强提供了一条途径，将基于扩散的增强器的众多反向步骤压缩为少数几步。当实例化在薛定谔桥（SB）上时，该桥将生成过程固定到干净的噪声端点，现有的一致性轨迹增强器（SBCTMs）仍然需要预训练的教师模型来提供轨迹监督，这增加了训练成本，并将最终质量与教师模型的质量绑定。我们提出了一个无教师、自蒸馏的一致性轨迹框架，移除了外部教师：轨迹目标由学生模型的指数移动平均（EMA）副本生成，模型通过三阶段课程训练，包括$x_0$预测、自蒸馏捷径目标以及使用多分辨率短时傅里叶变换（MR-STFT）损失的感知微调。使用与SBCTM相同的NCSN++骨干网络，我们的模型在VoiceBank+DEMAND数据集上无需教师模型即可达到宽带PESQ 3.01、ESTOI 0.87和SI-SDR 19.07 dB。通过改变步数和推理调度，我们发现低反向步数下的几何调度最大化感知质量，而高步数均匀调度有利于信号保真度，几何优势随反向步数增加而缩小。

英文摘要

Consistency trajectory models offer a route to fast, high- quality speech enhancement, collapsing the many reverse steps of diffusion-based enhancers into a handful. When instantiated on a Schrödinger bridge (SB), which pins the generative process to fixed clean and noisy endpoints, exist- ing consistency-trajectory enhancers (SBCTMs) still require a pretrained teacher to supply trajectory supervision, which raises training cost and ties the final quality to that of the teacher. We propose a teacher-free, self-distilled consistency- trajectory framework that removes the external teacher result- ing in a 5X reduction in per epoch training time. Our model is trained with a three-stage curriculum of clean speech pre- diction, a self-distilled shortcut objective, and perceptual fine-tuning with a multi-resolution short-time Fourier trans- form (MR-STFT) loss. Using the same NCSN++ backbone as SBCTM, our model attains a wide-band PESQ of 3.01, ES- TOI 0.87, and SI-SDR 19.07 dB on VoiceBank+DEMAND compared to 3.57, 0.87 and 12.8 dB for the teacher based model. Further, we find that a geometric schedule at low reverse step count maximizes perceptual quality, while a higher-step uniform schedule favors signal fidelity.

发表机构

• Institute of Systems Research, Dept. of Electrical
• Computer Engineering University of Maryland, College Park, MD, USA

机构由 AI 辅助整理，请以论文原文为准。