毫米波雷达感知的双条件GAN用于低信噪比信号的语音重建
mmWave Radar Aware Dual-Conditioned GAN for Speech Reconstruction of Signals With Low SNR
AI总结：
本文提出了一种雷达感知的双条件GAN，用于在低信噪比条件下实现毫米波信号的语音重建，通过多梅尔判别器和残差融合门提升生成器性能。
摘要：
毫米波雷达捕获的信号具有带宽受限和噪声大的特点，这使得对可理解的全带宽语音的重建变得困难。在本文中，我们提出了一种用于毫米波的两阶段语音重建流程，采用雷达感知的双条件生成对抗网络（RAD-GAN），能够对信噪比较低（-5 dB至-1 dB）的信号进行带宽扩展，这些信号通过玻璃墙壁捕获。我们提出了一种针对毫米波的多梅尔判别器（MMD）和残差融合门（RFG），以增强生成器的输入，以处理多个条件通道。所提出的两阶段流程包括在合成剪裁干净语音上预训练模型，然后在RFG生成的融合梅尔频谱图上进行微调。我们实证表明，所提出的方法在有限数据集上训练，没有预训练模块和数据增强，优于该特定任务的最先进方法。RAD-GAN的音频示例可在https://rad-gan-demo-site.vercel.app/上在线查看。
英文摘要：
Millimeter-wave (mmWave) radar captures are band-limited and noisy, making for difficult reconstruction of intelligible full-bandwidth speech. In this work, we propose a two-stage speech reconstruction pipeline for mmWave using a Radar-Aware Dual-conditioned Generative Adversarial Network (RAD-GAN), which is capable of performing bandwidth extension on signals with low signal-to-noise ratios (-5 dB to -1 dB), captured through glass walls. We propose an mmWave-tailored Multi-Mel Discriminator (MMD) and a Residual Fusion Gate (RFG) to enhance the generator input to process multiple conditioning channels. The proposed two-stage pipeline involves pretraining the model on synthetically clipped clean speech and finetuning on fused mel spectrograms generated by the RFG. We empirically show that the proposed method, trained on a limited dataset, with no pre-trained modules, and no data augmentations, outperformed state-of-the-art approaches for this specific task. Audio examples of RAD-GAN are available online at https://rad-gan-demo-site.vercel.app/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
