Toward Natural Emotional Text-To-Speech System with Fine-Grained Non-Verbal Expression Control
Nara Institute of Science and Technology(奈良先端科学技术大学院大学)
Wangzixi Zhou, Bagus Tris Atmaja, Sakriani Sakti
备注:
2025 28th Conference of the Oriental COCOSDA International Committee for the Co-ordination and Standardisation of Speech Databases and Assessment Techniques (O-COCOSDA)
发表信息:
Proc. 2025 28th Conference of the Oriental COCOSDA International Committee for the Co-ordination and Standardisation of Speech Databases and Assessment Techniques (O-COCOSDA), pp. 1-6, 2025
While current emotional Text-to-Speech (TTS) models have successfully controlled verbal prosody, they often ignore non-verbal vocalizations (NVs), which are essential for authentic human emotion. Although some non-verbal datasets have recently emerged, they often lack high-quality, fine-grained annotations, which restricts a model's ability to precisely control NV generation. To address this limitation, we propose a novel approach for fine-grained non-verbal expression synthesis. We curate and reprocess female NV utterances from the EARS corpus, develop a new annotation scheme using tags to encode NV types, frequencies, and durations, and build an emotional TTS benchmark to demonstrate its effectiveness. Our evaluation shows that while our NV approach leads to minor trade-offs in perceived naturalness, it significantly improves expressiveness (eMOS 4.20) and emotional recognition accuracy (78.8%). Emotion-specific analysis further reveals that NV cues are highly effective for high-arousal emotions like happy (82.5%) and fear (82.7%), and almost perfectly convey sadness (98.3%).
CARE: A Multimodal Corpus for Studying Speech and Non-Verbal Communication Across Multiple Medical Conditions
PRHLT, Universitat Politècnica de València(巴伦西亚理工大学);Sword Health(剑健康);INESC-ID(葡萄牙信息与通信系统研究所);Instituto Superior Técnico, Universidade de Lisboa(里斯本大学技术学院)
Proceedings of the ACII Affective Vocal Bursts Workshop and Competition 2022 (A-VB): Understanding a critically understudied modality of emotional expression
Attention-based Interactive Disentangling Network for Instance-level Emotional Voice Conversion
School of Computer Science and Engineering, Sun Yat-Sen University(中山大学计算机科学与工程学院);Guangdong Province Key Laboratory of Information Security Technology(广东省信息安全技术重点实验室);Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education(教育部机器智能与先进计算重点实验室)
AffectSpeech: A Large-Scale Emotional Speech Dataset with Fine-Grained Textual Descriptions for Speech Emotion Captioning and Synthesis
School of Biological Science and Medical Engineering, Southeast University(东南大学生物科学与医学工程学院);Key Laboratory of Child Development and Learning Science (Southeast University), Ministry of Education(教育部儿童发展与学习科学重点实验室(东南大学));Chair of Health Informatics (CHI), Technical University of Munich(慕尼黑工业大学健康信息学教席);Group on Language, Audio, & Music (GLAM), Imperial College London(帝国理工学院语言、音频与音乐组);Shenzhen Loop Area Institute(深圳河套学院);Shenzhen Research Institute of Big Data, School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen(深圳市大数据研究院,香港中文大学(深圳)人工智能学院)
Emotion Intensity and its Control for Emotional Voice Conversion
National University of Singapore(新加坡国立大学);Singapore University of Technology and Design(新加坡科技设计大学);University of Southern Queensland(南昆士兰大学);Imperial College London(帝国理工学院);University of Augsburg(奥格斯堡大学);University of Bremen(不来梅大学)
Fine-Grained Quantitative Emotion Editing for Speech Generation
School of Data Science(数据科学学院);Shenzhen Research Institute of Big Data(深圳大数据研究院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Alibaba Group(阿里巴巴集团)
Maestro-EVC: Controllable Emotional Voice Conversion Guided by References and Explicit Prosody
Graduate School of Artificial Intelligence(人工智能研究生院);Dept. of Computer Science and Engineering(计算机科学与工程系);Pohang University of Science and Technology (POSTECH)(釜山科学技术大学)
PromptEVC: Controllable Emotional Voice Conversion with Natural Language Prompts
Huawei Translation Service Center(华为翻译服务中心);Key Laboratory of Child Development and Learning Science (Southeast University)(儿童发展与学习科学重点实验室(东南大学));School of Biological Science and Medical Engineering(生物科学与医学工程学院)
MIKU-PAL: An Automated and Standardized Multi-Modal Method for Speech Paralinguistic and Affect Labeling
Santa Clara, CA, USA(美国圣克拉拉);Carnegie Mellon University(卡内基梅隆大学);Huazhong University of Science and Technology(华中科技大学);Nanjing University of Information Science and Technology(南京信息工程大学)
Enriching Multimodal Sentiment Analysis through Textual Emotional Descriptions of Visual-Audio Content
Tianjin University(天津大学);Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)(广东人工智能与数字经济实验室(深圳));Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)