International Audio Laboratories Erlangen（埃尔朗根国际音频实验室）；Friedrich Alexander University Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希·亚历山大大学）；Fraunhofer IIS（弗劳恩霍夫集成电路研究所）
Mohamed Elminshawi, Wolfgang Mack, Srikanth Raj Chetupalli, Soumitro Chakrabarty, Emanuël A. P. Habets
Speaker extraction (SE) aims to segregate the speech of a target speaker from a mixture of interfering speakers with the help of auxiliary information. Several forms of auxiliary information have been employed in single-channel SE, such as a speech snippet enrolled from the target speaker or visual information corresponding to the spoken utterance. The effectiveness of the auxiliary information in SE is typically evaluated by comparing the extraction performance of SE with uninformed speaker separation (SS) methods. Following this evaluation protocol, many SE studies have reported performance improvement compared to SS, attributing this to the auxiliary information. However, such studies have been conducted on a few datasets and have not considered recent deep neural network architectures for SS that have shown impressive separation performance. In this paper, we examine the role of the auxiliary information in SE for different input scenarios and over multiple datasets. Specifically, we compare the performance of two SE systems (audio-based and video-based) with SS using a common framework that utilizes the recently proposed dual-path recurrent neural network as the main learning machine. Experimental evaluation on various datasets demonstrates that the use of auxiliary information in the considered SE systems does not always lead to better extraction performance compared to the uninformed SS system. Furthermore, we offer insights into the behavior of the SE systems when provided with different and distorted auxiliary information given the same mixture input.
USED: Universal Speaker Extraction and Diarization
Shenzhen Research Institute of Big Data（深圳大数据研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；National University of Singapore（新加坡国立大学）；Saw Swee Hock School of Public Health, National University of Singapore（新加坡国立大学苏瑞福公共卫生学院）；Shanghai Jiao Tong University（上海交通大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
An Investigation on Speaker Augmentation for End-to-End Speaker Extraction
Beijing University of Posts and TelecommunicationsChina（北京邮电大学中国）；Tsinghua UniversityChina（清华大学中国）；School of Artificial Intelligence（人工智能学院）；Center for Speech and Language Technologies, BNRist（语音与语言技术中心）
School of Computer Science, Wuhan University（武汉大学计算机学院）；Suzhou Municipal Key Laboratory of Multimodal Intelligent Systems, Digital Innovation Research Center, Duke Kunshan University（多模态智能系统苏州市重点实验室、创新数字研究中心、杜克昆山大学）
Multi-Level Speaker Representation for Target Speaker Extraction
Northeastern University（东北大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；The Hong Kong Polytechnic University（香港理工大学）；Tencent（腾讯）；University of Bremen（不来梅大学）；National University of Singapore（新加坡国立大学）
SpEx+: A Complete Time Domain Speaker Extraction Network
Tianjin University（天津大学）；Nanyang Technological University（南洋理工大学）；Japan Advanced Institute of Science and Technology（日本北陆先端科学技术大学院大学）；National University of Singapore（新加坡国立大学）；University of Bremen（不来梅大学）
Listen to Extract: Onset-Prompted Target Speaker Extraction
Department of Computer Science, Inner Mongolia University（内蒙古大学计算机科学系）；Department of Computer Science and Engineering, Southern University of Science and Technology（南方科技大学计算机科学与工程系）；School of Automation and Intelligent Manufacturing, Southern University of Science and Technology（南方科技大学自动化与智能制造学院）
Improved Feature Extraction Network for Neuro-Oriented Target Speaker Extraction
School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；National University of Singapore（新加坡国立大学）；University of Science and Technology of China (USTC)（中国科学技术大学）
Universal Speaker Embedding Free Target Speaker Extraction and Personal Voice Activity Detection
School of Computer Science, Wuhan University, Wuhan, China（武汉大学计算机学院）；Suzhou Municipal Key Laboratory of Multimodal Intelligent Systems, Digital Innovation Research Center, Duke Kunshan University, Kunshan, China（多模态智能系统苏州市重点实验室、数字创新研究中心、杜克昆山大学）
ImagineNET: Target Speaker Extraction with Intermittent Visual Cue through Embedding Inpainting
National University of Singapore（新加坡国立大学）；University of Bremen（不来梅大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）
Nanjing University（南京大学）；Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Brno University of Technology（布拉格技术大学）；Northwestern Polytechnical University（西北工业大学）；NTT, Inc.（NTT公司）；Shanghai Jiao Tong University（上海交通大学）
Audio-Visual Target Speaker Extraction with Reverse Selective Auditory Attention
National University of Singapore（新加坡国立大学）；University of Science and Technology Beijing（北京科技大学）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Shenzhen Research Institute of Big data（深圳大数据研究院）；University of Bremen（不来梅大学）
WeSep: A Scalable and Flexible Toolkit Towards Generalizable Target Speaker Extraction
Shenzhen Research Institute of Big Data（深圳大数据研究院）；School of Data Science, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)（香港中文大学（深圳）数据科学学院）；Shanghai Jiao Tong University（上海交通大学）；Tencent AI Lab（腾讯人工智能实验室）
Dialogue-Based Streaming Audio-Visual Target Speaker Extraction with Predictive Dialogue Information
Shenzhen Loop Area Institute（深圳河套学院）；School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）人工智能学院）；The Chinese University of Hong Kong（香港中文大学）
