Deep learning-based speech enhancement (SE) models have achieved impressive performance in the past decade. Numerous advanced architectures have been designed to deliver state-of-the-art performance; however, their scalability potential remains unrevealed. Meanwhile, the majority of research focuses on small-sized datasets with restricted diversity, leading to a plateau in performance improvement. In this paper, we aim to provide new insights for addressing the above issues by exploring the scalability of SE models in terms of architectures, model sizes, compute budgets, and dataset sizes. Our investigation involves several popular SE architectures and speech data from different domains. Experiments reveal both similarities and distinctions between the scaling effects in SE and other tasks such as speech recognition. These findings further provide insights into the under-explored SE directions, e.g., larger-scale multi-domain corpora and efficiently scalable architectures.
AffectSpeech: A Large-Scale Emotional Speech Dataset with Fine-Grained Textual Descriptions for Speech Emotion Captioning and Synthesis
School of Biological Science and Medical Engineering, Southeast University(东南大学生物科学与医学工程学院);Key Laboratory of Child Development and Learning Science (Southeast University), Ministry of Education(教育部儿童发展与学习科学重点实验室(东南大学));Chair of Health Informatics (CHI), Technical University of Munich(慕尼黑工业大学健康信息学教席);Group on Language, Audio, & Music (GLAM), Imperial College London(帝国理工学院语言、音频与音乐组);Shenzhen Loop Area Institute(深圳河套学院);Shenzhen Research Institute of Big Data, School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen(深圳市大数据研究院,香港中文大学(深圳)人工智能学院)
DiM-Gestor: Co-Speech Gesture Generation with Adaptive Layer Normalization Mamba-2
School of Media Engineering, Communication University of Zhejiang(浙江传媒学院媒体工程学院);School of Broadcast Announcing Arts, Communication University of Zhejiang(浙江传媒学院播音主持艺术学院);Faculty of Humanities and Arts, Macau University of Science and Technology(澳门科技大学人文艺术学院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Zhejiang Lab(之江实验室);Changjiang Academy of Art and Design, Shantou University(汕头大学长江艺术与设计学院)
DOTA-ME-CS: Daily Oriented Text Audio-Mandarin English-Code Switching Dataset
Imperial College London(帝国理工学院);University of St Andrews(圣安德鲁斯大学);North China Electric Power University(华北电力大学);Technical University of Munich(慕尼黑工业大学);Wuhan University of Bioengineering(武汉生物工程学院)