This paper investigates the use of relative cues for text-based target speech extraction (TSE). We first provide a theoretical justification for relative cues from the perspectives of human perception and label quantization, showing that relative cues preserve fine-grained distinctions that are often lost in absolute categorical representations for continuous-valued attributes. Building on this analysis, we propose a two-stage TSE framework in which a speech separation model first generates candidate sources, followed by a text-guided classifier that selects the target speaker based on embedding similarity. Within this framework, we train two separate classification models to evaluate the advantages of relative cues over independent cues in case of continuous-valued attributes, considering both classification accuracy and TSE performance. Experimental results show that (i) relative cues achieve higher overall classification accuracy and improved TSE performance compared with independent cues; (ii) the proposed two-stage framework substantially outperforms single-stage text-conditioned extraction methods on both signal-level and objective perceptual metrics; and (iii) in controlled target-selection comparisons, the all cues, random cues, and temporal order settings outperform the strong enrollment-audio selection based on ECAPA-TDNN speaker embeddings, while language and distance cues remain competitive. In the WHAM!-noise evaluation, textual cues also show strong noise robustness, while all cues, random cues, and gender are especially robust. These results demonstrate the practical value of textual cues when enrollment speech is unavailable or cannot be used due to privacy concerns. Further analysis reveals notable differences in discriminative power across cue types, providing insights into the effectiveness of different relative cues for TSE.
MoMuSE: Momentum Multi-modal Target Speaker Extraction for Real-time Scenarios with Impaired Visual Cues
The Hong Kong Polytechnic University(香港理工大学);Shenzhen Research Institute of Big Data(深圳大数据研究院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
pTSE-T: Presentation Target Speaker Extraction using Unaligned Text Cues
University of Science and Technology Beijing (USTB), China(科学技术大学(USTB));Tongyi Lab, Alibaba Group, Singapore(通义实验室,阿里巴巴集团,新加坡);Hong Kong University of Science and Technology (HKUST), China(香港科技大学(HKUST))
CineSRD: Leveraging Visual, Acoustic, and Linguistic Cues for Open-World Visual Media Speaker Diarization
Hujing Digital Media and Entertainment Group(华景数字媒体与娱乐集团);MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所MAIS部门);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);School of Computer Science and Technology, Beijing Jiaotong University(北京交通大学计算机科学与技术学院)
Detect, Attend and Extract: Keyword Guided Target Speaker Extraction
Nanjing University(南京大学);X-LANCE Lab, MoE Key Lab of Artificial Intelligence, School of Computer Science(X-LANCE实验室,人工智能教育部重点实验室,计算机学院);ALTA Institute, Machine Intelligence Lab, Department of Engineering, University of Cambridge(剑桥大学工程系机器智能研究所ALTA研究所);National University of Singapore(新加坡国立大学);Shenzhen Loop Area Institute(深圳河套学院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shanghai Jiao Tong University(上海交通大学)
Token-level Speaker Change Detection Using Speaker Difference and Speech Content via Continuous Integrate-and-fire
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Bytedance AI LAB(字节跳动人工智能实验室)
Tianjin University(天津大学);College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);National University of Singapore(新加坡国立大学);Japan Advanced Institute of Science and Technology(日本北陆先端科学技术大学院大学)
Adaptive Speaker Embedding Self-Augmentation for Personal Voice Activity Detection with Short Enrollment Speech
College of Information Science and Technology, Donghua University(东华大学信息科学与技术学院);AI Research Center, Midea Group (Shanghai) Co.,Ltd.(美的集团(上海)有限公司人工智能研究中心)
WASE: Learning When to Attend for Speaker Extraction in Cocktail Party Environments
Institute of Automation, Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所);School of Future Technology, University of Chinese Academy of Sciences(中国科学院大学未来技术学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Center for Excellence in Brain Science and Intelligence Technology, CAS(中国科学院脑科学与智能技术卓越创新中心)
Adapting General Disentanglement-Based Speaker Anonymization for Enhanced Emotion Preservation
Singapore Institute of Technology(新加坡理工学院);Key Laboratory of Speech Acoustics and Content Understanding, Institute of Acoustics, Chinese Academy of Sciences(中国科学院声学研究所语音声学与内容理解重点实验室);National Institute of Informatics(日本国立信息研究所);Inria, Centre Inria de l’Université de Lorraine(法国国家信息与自动化技术研究院(Inria)卢瓦尔大学中心)
RadarVox: Radar-Audio Multimodal Cocktail-Party Speech Separation with Speaker-Aware Cross-Modal Matching
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Beijing University of Posts and Telecommunications(北京邮电大学);Dalian Maritime University(大连海事大学)
G-STAR: End-to-End Global Speaker-Tracking Attributed Recognition
Nanjing University(南京大学);Shanghai Jiao Tong University(上海交通大学);Central Media Technology Institute, Huawei(华为中央媒体技术研究院);Shenzhen Research Institute of Big Data(深圳大数据研究院);ETH Zürich(苏黎世联邦理工学院)
Robust Audio-Visual Target Speaker Extraction with Emotion-Aware Multiple Enrollment Fusion
School of Computer Science, Wuhan University, Wuhan, China(1 武汉大学计算机学院,武汉,中国);School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen, China(2 香港中文大学(深圳)人工智能学院,中国);School of Artificial Intelligence, Wuhan University, Wuhan, China(3 武汉大学人工智能学院,武汉,中国);School of Cyber Science and Engineering, Wuhan University, Wuhan, China(4 武汉大学网络科学与工程学院,武汉,中国);Digital Innovation Research Center, Duke Kunshan University, Kunshan, China(5 香港中文大学(深圳)数字创新研究中心,中国);AI Center, OPPO, Beijing, China(6 OPPO人工智能中心,北京,中国)