arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2606.29575cs.SDcs.AI

TF-MoE:用于高效语音分离的时频混合专家

TF-MoE: Time-Frequency Mixture-of-Experts for Efficient Speech Separation

Auditory Cognition and Computational Acoustics Lab Shanghai Jiao Tong University(上海交通大学听觉认知与计算声学实验室);Microsoft Research Asia(微软亚洲研究院)

Qinzhe Hu, Chenda Li, Wangyou Zhang, Shujie Liu, Yan Lu, Yanmin Qian

AI总结:

提出TF-MoE框架,通过时频维度的稀疏混合专家动态选择专家,在几乎不增加推理成本下提升模型容量,在低计算量下实现优于BSRNN的语音分离性能。

摘要:

近期语音分离(SS)的进展催生了参数规模紧凑的前端模型,但其高计算成本仍是部署到边缘设备的主要障碍。为解决此问题,我们提出TF-MoE,一种稀疏混合专家(MoE)框架,在几乎不增加推理成本的情况下增强模型容量。我们的方法通过交替的时间维度和频率维度MoE模块,在时间和频率维度引入动态专家专业化,每个模块为每帧或每个梅尔频带动态选择专家。基于梅尔频带分割的Conformer骨干网络,TF-MoE在低计算量设置下实现了强大的SS性能。实验结果表明,TF-MoE在计算成本约束下持续提升分离性能,在Libri2Mix上以相当的4.1 GMACs/s推理成本,比BSRNN高出+3.8 dB SDR。这使得TF-MoE成为边缘设备部署的有前景候选方案。

英文摘要:

Recent advances in speech separation (SS) have led to compact front-end models with small parameter sizes, yet their high computational cost remains a major barrier for deployment on edge devices. To address this, we propose TF-MoE, a sparse Mixture-of-Experts (MoE) framework that enhances model capacity with almost no increase in inference cost. Our method introduces dynamic expert specialization in time and frequency dimensions through alternating time-wise and frequency-wise MoE modules, each dynamically selecting experts per frame or mel band. Built upon a mel-band-splitting Conformer backbone, TF-MoE achieves strong performance on SS tasks under low-compute settings. Experimental results demonstrate that TF-MoE consistently improves separation performance under computation cost constraints, outperforming BSRNN by +3.8 dB SDR on Libri2Mix with comparable 4.1 GMACs/s inference cost. This positions TF-MoE as a promising candidate for edge-device deployment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑