arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.09277eess.AScs.CLcs.LG
收藏

面向低延迟语音匿名化的端到端流式模型

End-to-end streaming model for low-latency speech anonymization

Waris Quamer, Ricardo Gutierrez-Osuna

AI总结:

针对现有语音匿名化方法计算资源需求高、难适配实时流式场景的问题,提出基于端到端自编码器的低延迟流式匿名化模型,通过三类解耦表示合成语音,精简版在66ms延迟下仍保持最先进的自然度、可懂度与隐私保护性能。

摘要:

说话人匿名化旨在隐藏说话人身份线索的同时保留语言内容。当前基于机器学习的方法需要大量计算资源,阻碍了实时流式应用的发展。为解决这些问题,我们提出一种流式模型,可实现低延迟的说话人匿名化。该系统以端到端自编码器方式训练,包含提取类HuBERT信息的轻量级内容编码器、提取说话人身份的预训练说话人编码器,以及注入音高和能量信息的方差编码器。这三种解耦表示被送入解码器以重新合成语音信号。我们展示了系统两种实现的评估结果:完整模型延迟为230ms;精简版本(大小为完整模型的0.1倍)将延迟进一步降至66ms,同时在自然度、可懂度和隐私保护方面保持最先进性能。

英文摘要:

Speaker anonymization aims to conceal cues to speaker identity while preserving linguistic content. Current machine learning based approaches require substantial computational resources, hindering real-time streaming applications. To address these concerns, we propose a streaming model that achieves speaker anonymization with low latency. The system is trained in an end-to-end autoencoder fashion using a lightweight content encoder that extracts HuBERT-like information, a pretrained speaker encoder that extract speaker identity, and a variance encoder that injects pitch and energy information. These three disentangled representations are fed to a decoder that re-synthesizes the speech signal. We present evaluation results from two implementations of our system, a full model that achieves a latency of 230ms, and a lite version (0.1x in size) that further reduces latency to 66ms while maintaining state-of-the-art performance in naturalness, intelligibility, and privacy preservation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑