arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.01673cs.SDeess.AS
收藏

面向神经导向目标说话人提取的改进特征提取网络

Improved Feature Extraction Network for Neuro-Oriented Target Speaker Extraction

School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院);National University of Singapore(新加坡国立大学);University of Science and Technology of China (USTC)(中国科学技术大学)

Cunhang Fan, Youdian Gao, Zexu Pan, Jingjing Zhang, Hongyu Zhang, Jie Zhang, Zhao Lv

AI总结:

本文提出IFENet,通过双路径Mamba语音编码器和KAN脑电编码器提取特征,利用EEG注意力信息定位目标说话人,在KUL和AVED数据集上取得显著性能提升。

摘要:

听觉注意力解码(AAD)的近期快速发展为利用脑电图(EEG)作为目标说话人提取的辅助信息提供了可能。然而,有效建模长语音序列并从EEG信号中解析目标说话人身份仍然是一个重大挑战。本文提出了一种用于神经导向目标说话人提取的改进特征提取网络(IFENet),该网络主要由采用双路径Mamba的语音编码器和采用Kolmogorov-Arnold网络(KAN)的EEG编码器组成。我们提出了SpeechBiMamba,利用双路径Mamba对局部和全局语音序列进行建模以提取语音特征。此外,我们提出了EEGKAN,以有效提取与听觉刺激密切相关的EEG特征,并通过受试者的注意力信息定位目标说话人。在KUL和AVED数据集上的实验表明,IFENet优于最先进的模型,在开放评估条件下,在尺度不变信噪比(SI-SDR)方面分别取得了36%和29%的相对改进。

英文摘要:

The recent rapid development of auditory attention decoding (AAD) offers the possibility of using electroencephalography (EEG) as auxiliary information for target speaker extraction. However, effectively modeling long sequences of speech and resolving the identity of the target speaker from EEG signals remains a major challenge. In this paper, an improved feature extraction network (IFENet) is proposed for neuro-oriented target speaker extraction, which mainly consists of a speech encoder with dual-path Mamba and an EEG encoder with Kolmogorov-Arnold Networks (KAN). We propose SpeechBiMamba, which makes use of dual-path Mamba in modeling local and global speech sequences to extract speech features. In addition, we propose EEGKAN to effectively extract EEG features that are closely related to the auditory stimuli and locate the target speaker through the subject's attention information. Experiments on the KUL and AVED datasets show that IFENet outperforms the state-of-the-art model, achieving 36\% and 29\% relative improvements in terms of scale-invariant signal-to-distortion ratio (SI-SDR) under an open evaluation condition.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑