Spatial-Magnifier:用于多通道语音增强的空间上采样
Spatial-Magnifier: Spatial upsampling for multichannel speech enhancement
AI总结:
提出Spatial-Magnifier网络,从有限真实麦克风生成虚拟麦克风信号,结合SARL框架提升下游语音增强性能,接近全麦克风理想效果。
摘要:
虽然多通道语音增强算法的空间指向性随着麦克风数量的增加而提高,但将大型捕获阵列安装到实际边缘设备中通常受到物理限制。为了克服这一限制,我们提出了Spatial-Magnifier,一种神经网络,旨在从有限的真实麦克风(RM)测量中生成虚拟麦克风(VM)信号。此外,我们引入了空间音频表示学习(SARL)框架,该框架利用估计的VM信号和特征来调节下游语音增强系统。实验结果表明,所提出的框架在各种语音提取系统中优于现有的空间上采样基线,包括端到端多通道语音增强和神经波束形成。所提出的方法几乎恢复了所有麦克风可用时达到的理想性能。
英文摘要:
While the spatial directivity of multichannel speech enhancement algorithms improves with the number of microphones, fitting large capture arrays into real-world edge devices is typically limited by physical constraints. To overcome this limitation, we propose Spatial-Magnifier, a neural network designed to generate virtual microphone (VM) signals from a limited set of real microphone (RM) measurements. Moreover, we introduce the Spatial Audio Representation Learning (SARL) framework, which leverages estimated VM signals and features to condition a downstream speech enhancement system. Experimental results demonstrate that the proposed framework outperforms existing spatial upsampling baselines across various speech extraction systems, including end-to-end multichannel speech enhancement and neural beamforming. The proposed method nearly recovers the oracle performance achieved when all microphones are available.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。