VocalBridge: 基于扩散桥的潜在空间净化用于对抗基于扰动的语音指纹防御
VocalBridge: Latent Diffusion-Bridge Purification for Defeating Perturbation-Based Voiceprint Defenses
AI总结：
VocalBridge通过学习潜在空间映射实现高效净化，提升对抗基于扰动的语音指纹防御的鲁棒性。
摘要：
随着语音合成技术，包括文本到语音（TTS）和语音转换（VC）的快速发展，与语音克隆相关的安全和隐私问题变得更加严峻。最近的防御措施试图通过在语音中嵌入保护性扰动来防止未经授权的克隆，以模糊说话者身份同时保持可理解性。然而，攻击者可以应用先进的净化技术来去除这些扰动，恢复真实的音频特征，并重新生成可克隆的语音。尽管此类攻击的逼真度不断提高，但现有防御措施在对抗性净化下的鲁棒性仍缺乏充分研究。大多数现有的净化方法是为自动语音识别（ASR）系统中的对抗性噪声设计的，而不是说话人验证或语音克隆管道。因此，它们无法抑制定义说话者身份的细粒度音频线索，并且在说话人验证攻击（SVA）中往往无效。为了解决这些局限性，我们提出了扩散桥（VocalBridge），一种净化框架，它在EnCodec潜在空间中学习从扰动到清洁语音的潜在映射。使用带有余弦噪声调度的时序条件1D U-Net，该模型实现了高效的、无需转录文本的净化，同时保持说话者区分结构。我们进一步引入了受Whisper引导的音素变体，该变体结合了轻量级的时间引导，而无需使用真实的转录文本。实验结果表明，我们的方法在从受保护语音中恢复可克隆语音方面始终优于现有净化方法。我们的发现展示了当前基于扰动的防御措施的脆弱性，并突显了对不断演变的语音克隆和说话人验证威胁需要更鲁棒的保护机制的必要性。
英文摘要：
The rapid advancement of speech synthesis technologies, including text-to-speech (TTS) and voice conversion (VC), has intensified security and privacy concerns related to voice cloning. Recent defenses attempt to prevent unauthorized cloning by embedding protective perturbations into speech to obscure speaker identity while maintaining intelligibility. However, adversaries can apply advanced purification techniques to remove these perturbations, recover authentic acoustic characteristics, and regenerate cloneable voices. Despite the growing realism of such attacks, the robustness of existing defenses under adaptive purification remains insufficiently studied. Most existing purification methods are designed to counter adversarial noise in automatic speech recognition (ASR) systems rather than speaker verification or voice cloning pipelines. As a result, they fail to suppress the fine-grained acoustic cues that define speaker identity and are often ineffective against speaker verification attacks (SVA). To address these limitations, we propose Diffusion-Bridge (VocalBridge), a purification framework that learns a latent mapping from perturbed to clean speech in the EnCodec latent space. Using a time-conditioned 1D U-Net with a cosine noise schedule, the model enables efficient, transcript-free purification while preserving speaker-discriminative structure. We further introduce a Whisper-guided phoneme variant that incorporates lightweight temporal guidance without requiring ground-truth transcripts. Experimental results show that our approach consistently outperforms existing purification methods in recovering cloneable voices from protected speech. Our findings demonstrate the fragility of current perturbation-based defenses and highlight the need for more robust protection mechanisms against evolving voice-cloning and speaker verification threats.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
