零样本文本转语音中的目标说话人投毒框架
Speech Generation Speaker Poisoning: Capability Erasure in Zero-Shot Text-to-Speech
AI总结:
针对零样本TTS语音克隆的隐私风险,提出说话人生成投毒(SGSP)任务,通过修改训练模型阻止特定身份生成,并评估了推理时过滤和参数修改基线在1、15和100个遗忘说话人上的隐私-效用权衡。
摘要:
零样本文本转语音(TTS)语音克隆带来了严重的隐私风险,需要从训练好的TTS模型中移除特定说话人身份。传统的机器遗忘在此情境下不足,因为零样本TTS可以从仅参考提示动态重建声音。我们将此任务形式化为说话人生成投毒(SGSP),其中我们修改训练模型以防止生成特定身份,同时保留其他说话人的效用。我们评估了推理时过滤和参数修改基线在1、15和100个遗忘说话人上的表现。通过效用(WER)和隐私之间的权衡来评估性能,隐私使用AUC和遗忘说话人相似度(FSSIM)量化。我们在最多15个说话人上实现了强隐私,但由于身份重叠增加,在100个说话人时揭示了可扩展性限制。因此,我们的研究引入了一个新颖的问题和评估框架,以推动生成式语音隐私的进一步进展。
英文摘要:
Recent zero-shot Text-to-Speech (TTS) systems can clone previously unseen voices from only a few seconds of audio. We formulate Speech Generation Speaker Poisoning (SGSP), a task that seeks to prevent a model from synthesizing targeted speaker identities while maintaining performance on all other speakers. Unlike conventional machine unlearning, removing training examples is insufficient because modern zero-shot TTS systems can reconstruct identities through learned speaker representations and strong generalization capabilities. We evaluate both inference-time filtering and parameter-modification approaches across settings involving 1, 15, and 100 forget speakers, focusing primarily on speakers seen during training, which we show are harder to suppress than unseen speakers. To characterize the trade-off between utility and privacy, we introduce an evaluation framework based on AUC analysis and a proposed metric, Forget Set Similarity (FSSIM). Our results demonstrate effective speaker suppression for up to 15 forget speakers while revealing that worst-case identity leakage (Max-FSSIM) remains unresolved at multi-speaker scale - establishing this as an open challenge for future work. Together, our work establishes targeted speaker poisoning as a task for zero-shot TTS and, using StyleTTS2 as an initial testbed, provides methods and evaluation protocols for future research, with code and model weights.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。