arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.01469cs.SDcs.AIeess.AS
收藏

TIGER:用于高效语音分离的时间-频率交织增益提取与重建

TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separation

Department of Computer Science and Technology, Institute for AI, BNRist, Tsinghua University(计算机科学与技术系、人工智能研究院、BNRist、清华大学);Tsinghua Laboratory of Brain and Intelligence (THBI), IDG/McGovern Institute for Brain Research, Tsinghua University(脑智能实验室(THBI)、IDG/麦克戈文脑科学研究院、清华大学);Chinese Institute for Brain Research (CIBR), Beijing(中国脑科学研究院)

Mohan Xu, Kai Li, Guo Chen, Xiaolin Hu

AI总结:

TIGER通过时间-频率交织增益提取与重建技术,在降低参数和计算成本的同时,实现了超越SOTA模型的语音分离性能。

摘要:

近年来,许多语音分离研究主要集中在提高模型性能上。然而,对于低延迟语音处理系统,高效率同样至关重要。因此,我们提出了一种参数和计算成本显著降低的语音分离模型:时间-频率交织增益提取与重建网络(TIGER)。TIGER利用先验知识将频率带划分并压缩频率信息。我们采用多尺度选择性注意力模块来提取上下文特征,同时引入全频率帧注意力模块以捕捉时间和频率的上下文信息。此外,为了更真实地评估语音分离模型在复杂声学环境中的性能,我们引入了一个名为EchoSet的数据集。该数据集包含噪声和更真实的混响(例如考虑物体遮挡和材料属性),并包含两名说话人以随机比例重叠的语音。实验结果表明,训练在EchoSet上的模型相比在现实世界中收集的数据具有更好的泛化能力,这验证了EchoSet的实用价值。在EchoSet和现实数据上,TIGER将参数数量减少了94.3%,将MACs减少了95.3%,同时实现了超越当前最先进的模型TF-GridNet的性能。

英文摘要:

In recent years, much speech separation research has focused primarily on improving model performance. However, for low-latency speech processing systems, high efficiency is equally important. Therefore, we propose a speech separation model with significantly reduced parameters and computational costs: Time-frequency Interleaved Gain Extraction and Reconstruction network (TIGER). TIGER leverages prior knowledge to divide frequency bands and compresses frequency information. We employ a multi-scale selective attention module to extract contextual features while introducing a full-frequency-frame attention module to capture both temporal and frequency contextual information. Additionally, to more realistically evaluate the performance of speech separation models in complex acoustic environments, we introduce a dataset called EchoSet. This dataset includes noise and more realistic reverberation (e.g., considering object occlusions and material properties), with speech from two speakers overlapping at random proportions. Experimental results showed that models trained on EchoSet had better generalization ability than those trained on other datasets compared to the data collected in the physical world, which validated the practical value of the EchoSet. On EchoSet and real-world data, TIGER significantly reduces the number of parameters by 94.3% and the MACs by 95.3% while achieving performance surpassing the state-of-the-art (SOTA) model TF-GridNet.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑