arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2505.15004eess.AScs.SD

EASY:通过因子化蒸馏实现情感感知的说话人匿名化

EASY: Emotion-aware Speaker Anonymization via Factorized Distillation

Northwestern Polytechnical University(西北工业大学);Nanyang Technological University(南洋理工大学)

Jixun Yao, Hexin Liu, Eng Siong Chng, Lei Xie

AI总结:

提出EASY框架,通过顺序解耦和因子化蒸馏分离说话人身份、语言内容与情感表示,在保护隐私的同时保留情感状态,优于所有基线系统。

摘要:

情感在语音交互中起着重要作用,通过语调、音高和节奏传达,使表达超越字词以展现感受和意图,从而创造更个性化的体验。然而,现有大多数说话人匿名化系统采用并行解耦方法,仅将语音分离为语言内容和说话人身份,往往忽略了对原始情感状态的保留。本研究提出EASY,一个情感感知的说话人匿名化框架。EASY采用新颖的顺序解耦过程,分离说话人身份、语言内容和情感表示,通过因子化蒸馏方法在不同子空间中对每种语音属性进行建模。通过独立约束说话人身份和情感表示,EASY最小化信息泄漏,在保留原始语言内容和情感状态的同时增强隐私保护。在VoicePrivacy Challenge官方数据集上的实验结果表明,所提方法优于所有基线系统,在保持语言内容和情感状态的同时有效保护说话人隐私。

英文摘要:

Emotion plays a significant role in speech interaction, conveyed through tone, pitch, and rhythm, enabling the expression of feelings and intentions beyond words to create a more personalized experience. However, most existing speaker anonymization systems employ parallel disentanglement methods, which only separate speech into linguistic content and speaker identity, often neglecting the preservation of the original emotional state. In this study, we introduce EASY, an emotion-aware speaker anonymization framework. EASY employs a novel sequential disentanglement process to disentangle speaker identity, linguistic content, and emotional representation, modeling each speech attribute in distinct subspaces through a factorized distillation approach. By independently constraining speaker identity and emotional representation, EASY minimizes information leakage, enhancing privacy protection while preserving original linguistic content and emotional state. Experimental results on the VoicePrivacy Challenge official datasets demonstrate that our proposed approach outperforms all baseline systems, effectively protecting speaker privacy while maintaining linguistic content and emotional state.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑