Improved Feature Extraction Network for Neuro-Oriented Target Speaker Extraction
School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院);National University of Singapore(新加坡国立大学);University of Science and Technology of China (USTC)(中国科学技术大学)
The recent rapid development of auditory attention decoding (AAD) offers the possibility of using electroencephalography (EEG) as auxiliary information for target speaker extraction. However, effectively modeling long sequences of speech and resolving the identity of the target speaker from EEG signals remains a major challenge. In this paper, an improved feature extraction network (IFENet) is proposed for neuro-oriented target speaker extraction, which mainly consists of a speech encoder with dual-path Mamba and an EEG encoder with Kolmogorov-Arnold Networks (KAN). We propose SpeechBiMamba, which makes use of dual-path Mamba in modeling local and global speech sequences to extract speech features. In addition, we propose EEGKAN to effectively extract EEG features that are closely related to the auditory stimuli and locate the target speaker through the subject's attention information. Experiments on the KUL and AVED datasets show that IFENet outperforms the state-of-the-art model, achieving 36\% and 29\% relative improvements in terms of scale-invariant signal-to-distortion ratio (SI-SDR) under an open evaluation condition.
ExPO: Explainable Phonetic Trait-Oriented Network for Speaker Verification
National University of Singapore(新加坡国立大学);Shenzhen Research Institute of Big Data(深圳大数据研究院);School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院)
EEG-informed attended speaker extraction from recorded speech mixtures with application in neuro-steered hearing prostheses
ESAT Laboratory of KU Leuven(KU莱顿大学ESAT实验室);KU Leuven(KU莱顿大学);Department of Electrical Engineering (ESAT)(电气工程系(ESAT));Department of Neurosciences(神经科学系)
SpeakerCard-1M: An Evidence-Grounded Corpus for In-the-Wild Speaker Verification
Brno University of Technology, Czechia(布拉格技术大学,捷克);Peking University, China(北京大学,中国);Xiaomi, China(小米,中国);Athens University of Economics and Business, Greece(雅典经济与商业大学,希腊);The Hong Kong Polytechnic University, Hong Kong(香港理工大学,香港);Nanjing University, China(南京大学,中国)
Identifying Speaker Information in Feed-Forward Layers of Self-Supervised Speech Transformers
Graduate Institute of Communication Engineering, National Taiwan University(国立台湾大学通信工程研究所);Centre of Speech Technology Research, University of Edinburgh(爱丁堡大学语音技术研究中心)
1 College of Electronic;Optical Engineering \& College of Flexible Electronics (Future Technology), Nanjing University of Posts;Telecommunications, Nanjing, China 2 School of Artificial Intelligence, School of Data Science, SRIBD, The Chinese University of Hong Kong, Shenzhen, China
ASRRL-TTS: Agile Speaker Representation Reinforcement Learning for Text-to-Speech Speaker Adaptation
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Department of Automation, Tsinghua University(清华大学自动化系);Beijing National Research Center for Information Science and Technology, Tsinghua University(北京信息科学与技术国家研究中心);Kuaishou Technology Co., Ltd.(快手科技有限公司);Tianjin University(天津大学);Shanghai Polytechnic University(上海第二工业大学)
Structured Speaker-Deficiency Adaptation of Foundation Models for Dysarthric and Elderly Speech Recognition
The Chinese University of Hong Kong(香港中文大学);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);National Research Council of Canada(加拿大国家研究委员会)
Egocentric Speaker Classification in Child-Adult Dyadic Interactions: From Sensing to Computational Modeling
University of Southern California(南加州大学);University of California, San Francisco(加州大学旧金山分校);Signal Analysis and Interpretation Laboratory(信号分析与解释实验室);Weill Institute for Neurosciences(魏尔神经科学研究所)
Towards Intelligibility-Oriented Audio-Visual Speech Enhancement
Edinburgh Napier University(爱丁堡龙比亚大学)
现有的基于深度学习(DL)的语音增强方法通常被优化以最小化干净语音特征与增强语音特征之间的距离。这些通常导致改善的语音质量,但是它们缺乏概括性,并且在实际嘈杂的情况下可能无法提供所需的语音清晰度。为了应对这些挑战,研究人员已经探索了面向可懂度(I-O)的损失函数和视听(AV)信息的整合,以实现更鲁棒的语音增强(SE)。在本文中,我们介绍了基于DL的I-O SE算法利用AV信息,这是一个新的和以前未探索的研究方向。具体来说,我们提出了一个完全卷积的AV SE模型,该模型使用修改后的短时客观可懂度(STOI)度量作为训练成本函数。据我们所知,这是第一项利用AV模态与基于I-O的SE损失函数的集成的工作。比较实验结果表明,我们提出的I-O AV SE框架优于音频(AO)和AV模型训练与传统的基于距离的损失函数,在标准的客观评价措施,当处理看不见的扬声器和噪声。