TF-MoE:用于高效语音分离的时频混合专家
TF-MoE: Time-Frequency Mixture-of-Experts for Efficient Speech Separation
AI总结:
提出TF-MoE框架,通过时频维度的稀疏混合专家动态选择专家,在几乎不增加推理成本下提升模型容量,在低计算量下实现优于BSRNN的语音分离性能。
摘要:
近期语音分离(SS)的进展催生了参数规模紧凑的前端模型,但其高计算成本仍是部署到边缘设备的主要障碍。为解决此问题,我们提出TF-MoE,一种稀疏混合专家(MoE)框架,在几乎不增加推理成本的情况下增强模型容量。我们的方法通过交替的时间维度和频率维度MoE模块,在时间和频率维度引入动态专家专业化,每个模块为每帧或每个梅尔频带动态选择专家。基于梅尔频带分割的Conformer骨干网络,TF-MoE在低计算量设置下实现了强大的SS性能。实验结果表明,TF-MoE在计算成本约束下持续提升分离性能,在Libri2Mix上以相当的4.1 GMACs/s推理成本,比BSRNN高出+3.8 dB SDR。这使得TF-MoE成为边缘设备部署的有前景候选方案。
英文摘要:
Recent advances in speech separation (SS) have led to compact front-end models with small parameter sizes, yet their high computational cost remains a major barrier for deployment on edge devices. To address this, we propose TF-MoE, a sparse Mixture-of-Experts (MoE) framework that enhances model capacity with almost no increase in inference cost. Our method introduces dynamic expert specialization in time and frequency dimensions through alternating time-wise and frequency-wise MoE modules, each dynamically selecting experts per frame or mel band. Built upon a mel-band-splitting Conformer backbone, TF-MoE achieves strong performance on SS tasks under low-compute settings. Experimental results demonstrate that TF-MoE consistently improves separation performance under computation cost constraints, outperforming BSRNN by +3.8 dB SDR on Libri2Mix with comparable 4.1 GMACs/s inference cost. This positions TF-MoE as a promising candidate for edge-device deployment.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。