我们能通过毫米波雷达听到你!一个端到端的窃听系统
We Can Hear You with mmWave Radar! An End-to-End Eavesdropping System
AI总结:
本文提出mmSpeech端到端窃听系统,利用毫米波雷达感知扬声器振动穿透墙壁重建可懂语音,无需说话人先验知识,并通过深度神经网络和ASR编码器微调实现最先进语音质量与良好泛化。
摘要:
随着语音技术的兴起,扬声器播放已变得普遍,对语音隐私构成日益严重的风险。传统窃听方法通常需要侵入式接触或视线条件,限制了其实用性。本文提出mmSpeech,一个基于毫米波的端到端窃听系统,仅利用扬声器播放引起的振动信号即可重建可懂语音,甚至可穿透墙壁且无需说话人先验知识。为实现这一目标,我们揭示了在使用窄带毫米波信号捕获高质量振动时,振动材料与雷达采样率的最优组合。随后设计了一个深度神经网络,从估计的含噪声频谱图中重建可懂语音。为进一步支持下游语音理解,我们引入合成训练流程,并对预训练ASR模型的编码器进行选择性微调。我们使用商用毫米波雷达实现mmSpeech,并通过大量实验验证其性能。结果表明,mmSpeech达到了最先进的语音质量,并在未见过的说话人和多种条件下具有良好的泛化能力。
英文摘要:
With the rise of voice-enabled technologies, loudspeaker playback has become widespread, posing increasing risks to speech privacy. Traditional eavesdropping methods often require invasive access or line-of-sight, limiting their practicality. In this paper, we present mmSpeech, an end-to-end mmWave-based eavesdropping system that reconstructs intelligible speech solely from vibration signals induced by loudspeaker playback, even through walls and without prior knowledge of the speaker. To achieve this, we reveal an optimal combination of vibrating material and radar sampling rate for capturing high-quality vibrations using narrowband mmWave signals. We then design a deep neural network that reconstructs intelligible speech from the estimated noisy spectrograms. To further support downstream speech understanding, we introduce a synthetic training pipeline and selectively fine-tune the encoder of a pre-trained ASR model. We implement mmSpeech with a commercial mmWave radar and validate its performance through extensive experiments. Results show that mmSpeech achieves state-of-the-art speech quality and generalizes well across unseen speakers and various conditions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。